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ABSTRACT 

This  thesis  examines  whether  a  Decision  Support  System  (DSS)  could  be  of  practical 
assistance  to  the  U.  S.  Navy's  shipboard  Tactical  Action  Officer  (TAO).  The  environment 
of  the  TAO  is  one  of  tension  and  information  overload  as  he  monitors  the  data  coming  in 
from  sensors  and  other  watchstanders  and  attempts  to  determine  the  tactical  situation  from 
these  clues.  The  thesis  begins  with  the  assumption  that  in  order  to  help  the  TAO  perform 
his  duties  and  reach  a  correct  analysis  of  the  developing  tactical  picture,  a  knowledge-based 
DSS  could  be  developed  to  assist  him  by  figuring  out  the  identity  of,  and  associated  threat 
presented  by  each  contact  and  to  help  him  remember  what  the  identity  so  far  has  been. 
Through  a  series  of  interviews  of  qualified  TAOs,  a  set  of  requirements  for  such  a  system 
was  developed.  The  results  were  analyzed  and  the  subsequent  design  made  to  include 
analytic  capabilities,  Rules  of  Engagement  recommendations,  and  a  trackball  interface 
featuring  pull-down  menus  and  familiar  symbology. 
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I.     INTRODUCTION 

This  first  chapter  will  provide  the  reader  with  an  overview  of  the  duties  and 
environment  of  the  Tactical  Action  Officer  (TAO),  and  describe  how  a  Decision  Support 
System  (DSS)  may  be  of  potential  benefit  to  the  TAO. 

A.  BACKGROUND 

1 .      The  Environment  and  Duties  of  the  Tactical  Action  Officer 

The  environment  of  the  U.  S.  Navy's  Tactical  Action  Officer  (TAO)  is  one  of 
stress,  potential  information  overload,  doubt,  and  sometimes  confusion.  It  is  the  TAO  who 
is  charged  with  the  awesome  responsibility  of  fighting  the  ship,  employing  the  ship's 
weapons  against  aggressor  platforms  in  the  Captain's  absence. 

In  the  days  of  World  War  I,  the  Commanding  Officer  fought  his  ship  from  the 
bridge  where  he  visually  identified  the  enemy  and  decided  on  the  appropriate  engagement 
method.  Then  came  wars  fought  under  the  cover  of  darkness  —  wars  fought  with  weapons 
other  than  cannon  —  wars  fought  without  ever  sighting  the  enemy. 

Electronic  detection  became  prevalent  in  World  War  II  with  the  advent  of  surface 
search  and  air  search  radar  and  sonar.  These  devices  not  only  provided  non-visual  target 
detection  for  the  originator,  but  eventually  were  used  to  locate,  identify,  and  target  the 
emitting  platform. 

In  today's  high-tech  world  of  phased-array  radars,  electronic  surveillance 
methods,  smart  weapons,  and  multiple  threats,  war  at  sea  is  becoming  increasingly 
complex.  The  Commanding  Officer  afloat  needs  help  in  discharging  his  responsibilities,  he 


needs  a  highly  trained  watch  officer  prepared  to  advise  him  and  fight  the  ship  in  his 
absence.  Such  an  individual  is  the  TAO. 

When  engaged  in  fighting  the  ship  (sometimes  just  in  keeping  track  of  the  tactical 
picture),  the  TAO  experiences  stress  and  sometimes  an  overload  of  information.  Such 
stressful  situations  may  threaten  the  ship,  her  crew,  and  her  ability  to  carry  out  her  mission. 
2 .      Evolution  of  the  At-Sea  Tactical  Situation 

The  TAO  gathers  information  about  the  contacts  he  encounters:  size  information, 
data  about  their  electronic  emissions,  speed  information,  and  finally  configuration 
information  gained  through  visual  sighting  by  lookout  or  friendly  aircraft.  From  these 
inputs  the  TAO  tries  to  narrow  down  the  possible  identity  of  these  contacts  in  order  to  keep 
track  of  the  developing  tactical  situation. 

Despite  the  doubt  inherent  in  the  TAO  environment  mentioned  earlier,  some 
TAOs  will  stick  stubbornly  to  a  decision  they  have  made,  no  matter  how  unfounded  this 
decision  may  be.  This  phenomenon  is  probably  the  result  of  a  combination  of  factors,  lack 
of  sleep,  stress,  difficult  subordinates  whose  attitudes  makes  the  TAO  even  more  adamant, 
and  the  tendency  of  the  position  of  TAO  to  sometimes  force  the  decision  simply  because  he 
is  the  boss  (I'm  the  TAO,  and  if  I  say  it's  a  Krivak,  then  it's  a  Krivak).  This  can  be 
dangerous  and,  in  fact,  potentially  disastrous.  Staw  points  out  that  because  it  is  often 
possible  for  persons  who  have  suffered  a  setback  to  recoup  their  losses  through  an  even 
greater  commitment  of  resources  to  the  same  course  of  action,  a  cycle  of  escalating 
commitment  can  be  produced.  [Staw  81  p.  577]  By  design,  a  DSS  would  reduce/eliminate 
this  problem  by  offering  another  expert  opinion  and  by  allowing  the  TAO  a  graceful,  face- 
saving  way  out  of  a  bad  decision.  Computers,  after  all,  are  seldom  smart-alec  and  difficult 
to  get  along  with. 


It  is  conceivable  that  if  an  automated  decision  aid  had  existed  aboard  the  USS 
Stark  in  the  Persian  Gulf,  that  disastrous  situation  could  have  been  avoided.  From  the  time 
the  Stark  "was  warned  of  the  approach  of  an  Iraqi  F-l  Mirage  by  an  E-3  airborne  warning 
and  control  system  (AWACS)  aircraft"  [Vlahos  88  p.  64-65]  at  2000  local  time  until  the  F- 
l's  "first  missile  was  launched  at  2107,  22.5  nautical  miles  from  the  Stark"  the  aircraft  had 
closed  a  total  distance  of  about  175  nautical  miles,  "locked  onto  the  Stark"  with  its  Cyrano 
IV  air  intercept  radar,  changed  course  to  "a  virtual  constant  bearing,  decreasing  range"  path 
now  coming  right  at  the  Stark,  and  fired  two  Exocet  missiles.  Stark's  TAO  forbade 
contacting  the  aircraft  by  radio  until  after  the  first  missile  had  been  launched.  After  the 
second  missile  had  been  fired  the  Phalanx  Gatling  gun  was  placed  in  'standby  mode'"  and 
"permission  was  given  at  this  time  to  arm  the  super  rapid  blooming  offboard  chaff 
(SRBOC)  launchers."  Evidently,  despite  all  of  the  indications  of  the  F-l's  intentions,  his 
hostile  intent  was  never  recognized.  The  added  information  and  advice  a  DSS  could  have 
provided  might  have  led  to  a  better  decision  about  the  interpretation  of  the  tactical  situation 
at  hand  and  to  a  better  reaction  on  the  part  of  the  TAO  to  defend  his  ship. 

B  .   OBJECTIVES  OF  RESEARCH 

This  paper  describes  research  conducted  to  determine  whether  a  micro-  or  mini- 
computer-based Decision  Support  System  would  be  useful  to  the  TAO  in  analyzing  the 
input  data,  sorting  the  contacts  by  eliminating  those  which  do  not  fit  the  data,  and  keeping 
track  of  what  the  contact  could  possibly  be  and  presenting  this  information  to  the  TAO.  In 
effect,  the  system  would  act  as  an  "analytic  scratchpad." 


The  specific  objectives  of  this  thesis  work  were  to: 

•  determine  if  a  Knowledge-Based  DSS  could  be  helpful  to  the  TAO  for: 

a.  determining  the  identities  of  other  platforms 

b.  remembering  these  identities  for  the  TAO 

c.  recommending  courses  of  action 

If  the  above  proved  feasible,  to: 

•  analyze  the  specific  requirements  for  such  a  system 

•  design  a  system  to  meet  the  need 

C .  BENEFITS  OF  THE  STUDY 

The  advantages  of  such  a  system  arise  from  the  DSS's  ability  to  determine  from  the 
input  data  the  type  of  contact  and  to  keep  track  of  this  information  for  the  TAO  to  free  his 
mind  for  other  things. 

A  further  advantage  is  seen  in  the  very  nature  of  a  computer  --  it  is  dispassionate  and 
unemotional,  and  it  does  not  become  stressed.  By  allowing  it  to  make  these  cool,  calm, 
unbiased  analyses,  the  Navy  benefits  by  not  having  a  stress-induced  misjudgment  on  the 
part  of  some  TAO  cause  the  ship  to  take  a  missile  hit. 

D.  SCOPE,  LIMITATIONS,  AND  ASSUMPTIONS 

It  is  important  to  note  that  the  system  designed  was  set  up  not  as  a  reaction  tool  for  the 
crisis  environment  of  a  "shooting  war"  -  it  will  not  take  over  from  a  panic-stricken  TAO 


and  defend  the  ship  or  even  advise  him  when  the  chips  are  already  down.  Rather,  it  will 
act  as  an  advisor  during  the  DEVELOPING  tactical  situation.  It  will  take  data  input,  and 
evaluate  it  based  on  the  information  in  a  platform  database  which  meets  the  given  criteria. 

It  was  assumed  that  the  TAOs  at  the  U.  S.  Naval  Postgraduate  School  (NPS)  generally 
reflect  the  overall  opinions  of  the  TAOs  at  sea  and  that  the  school's  influence  on  these 
students  had  not  dulled  or  distorted  their  professional  opinions  due  to  their  absence  from 
the  fleet.  Further  research  might  be  required  to  confirm  that,  in  fact,  the  cutting  edge  TAOs 
of  the  fleet  do  agree  with  the  design  specifications  produced  as  a  result  of  this  survey. 

E.  RESEARCH  METHODOLOGY 

A  survey  of  those  qualified  TAOs  available  at  the  U.  S.  Naval  Postgraduate  School 
was  conducted  to  determine  what  they  thought  might  be  the  most  beneficial  means  of 
supporting  the  TAO  in  such  an  environment.  These  TAOs  were  interviewed  by  the  author 
with  a  guideline  set  of  questions  to  structure  the  interview  (Appendix  A).  These  guidelines 
were  developed  based  on  prior  research  in  the  DSS  arena,  although  little  of  this  research 
actually  dealt  with  real  world  situations.  Rather,  it  discussed  mostly  theoretical  applications 
to  a  proposed  area. 

Although  the  use  of  questionnaires  was  considered  for  those  unable  to  be  scheduled 
for  an  interview,  none  were  used,  due  to  the  extra  information  and  different  insights  more 
likely  to  evolve  in  an  interview  than  in  a  structured  questionnaire  and  due  to  the  complexity 
of  the  questions  themselves  and  the  possible  resulting  misinterpretation. 

F.  THESIS  ORGANIZATION 

The  thesis  is  organized  into  five  chapters.  This  first  chapter  is  the  introduction  which 
has  described  the  duties  and  environment  of  the  Tactical  Action  Officer  (TAO)  and  how  a 


DSS  may  be  of  potential  benefit  to  the  TAO.  The  second  chapter  explores  the  theory  behind 
Decision  Support  Systems  and  Expert  Systems  in  general  and  those  specifically  tailored  to 
the  emergency/crisis  environment  (previous  attempts  at  this  specific  problem  domain).  The 
third  chapter  describes  the  TAO  environment  as  it  relates  to  the  above  theoretical  basis,  the 
research  actually  conducted  as  it  related  to  this  environment  and  the  problem  presented. 
The  fourth  chapter  discusses  the  interview  results  and  the  design  of  the  resulting  system. 
Finally,  the  fifth  chapter  presents  further  potential  research  possibilities  and  discusses 
conclusions  and  recommendations  drawn  from  this  research. 


II.  THEORETICAL  FOUNDATIONS  FOR  THE  RESEARCH 

In  order  to  properly  answer  whether  the  Decision  Support  System  (DSS)  is 
appropriate  for  the  environment  of  the  TAO,  it  is  necessary  first  to  examine  the  theory 
behind  DSS  and  their  relatives  Expert  Systems  (ES)  and  Knowledge-Based  DSS.  It  is  also 
incumbent  on  a  researcher  in  this  field  to  take  a  serious  look  at  previous  attempts  to  assist 
the  crisis  manager  with  these  tools. 

A.   DECISION  SUPPORT  SYSTEMS 

Much  literature  exists  on  the  subject  of  Decision  Support  Systems  (DSS).  This  chapter 
condenses  the  essence  of  the  literature  to  describe  a  Decision  Support  System.  Such  an 
understanding  will  be  necessary  to  determine  whether  this  is  an  appropriate  solution  to  the 
problem  experienced  by  the  TAO.  The  results  come  from  a  variety  of  sources,  including 
actual  experiments  with  DSS  in  crisis  situations. 

There  is,  as  Hopple  points  out,  a  tendency  to  use  DSS  where  they  are  not  really 
appropriate.  He  compares  this  to  the  law  of  the  hammer,  "Give  a  child  a  hammer,  and  s/he 
will  use  it  on  everything  encountered."  He  warns  against  methidolatry,  which  manifests 
itself  at  the  most  general  level  as  the  fetishistic  belief  that  computer-based  decision  aids  are 
appropriate  solutions  to  (and  potential  panaceas  for)  every  analytical  and  decision  problem 
or  task  that  confronts  a  command  and  control  problem  solver.  [Hopple  86  p.  948-949) 
With  this  warning  in  mind,  an  examination  of  decision  aids  is  in  order. 

Ford  defines  a  Decision  Support  System  (DSS)  as  "something  that  helps  decision- 
makers utilize  data  and  models  to  solve  unstructured  or  semi-structured  problems."  [Ford 
85  p.  21-22].  It  is  a  highly  touted  tool  for  analyzing  problems  prior  to  decision  making. 


The  computer's  adeptness  at  manipulating  data  through  storage,  retrieval,  arithmetic 
operations,  and  comparison  makes  it  well  suited  to  the  operations  desired  by  humans. 

Sprague  and  Carlson  define  DSS  as  "computer-based  systems  that  help  decision 
makers  confront  ill- structured  problems  through  direct  interaction  with  data  and  analysis 
models."  [Sprague  &  Carlson  82  p.  1]  This  is  a  different  arena  from  that  of  Management 
Information  Systems  (MIS).  As  Sprague  points  out,  MIS  has  an  "information  focus, 
aimed  at  middle  managers.  Its  structured  information  flow  and  integration  by  function 
(i.e.,  production,  marketing,  etc.)  provide  inquiry  and  report  generation,  usually  with  a 
database."  [Sprague  87  p.  198]  With  a  DSS,  the  emphasis  is  on  enhancing  the  quality  of 
decisions. 

Though  each  of  these  definitions  seems  to  be  aiming  at  the  same  area  of  computer 
support,  there  appears  to  be  no  concrete  definition  of  a  Decision  Support  System  to  allow 
direct  comparison  and  contrasting  with  MIS.  This  thesis  will  use  a  more  general 
description  based  on  the  components  of  the  name,  due  to  the  simplicity  it  lends  to  the 
concept.  Keen  points  out  that  the  system  can  be  broken  into  its  component  parts  and 
described  in  terms  of  a  computer-based  system  to  support  the  making  of  decisions  [Keen 
87  p.  259].  A  distinction  should  be  made  here  ~  a  DSS  is  not  designed  to  replace  the 
decision-maker  but  rather  to  help  him  make  the  decision  by  presenting  the  information  in  a 
different  format  or  with  a  different  evaluation  of  the  situation. 

DSS  are  suitable  for  many  diverse  decision  areas  including  financial  planning,  train 
dispatching,  and  marketing  strategies.  They  are  particularly  adept  in  situations  requiring  a 
different  representation  or  a  sorting  of  data  prior  to  its  being  useful  to  the  decision-maker. 
This  different  representation  may  take  the  form  of  a  graphical  display  like  a  pie  chart  or  may 
involve  sensitivity  analysis  (What  if?)  capability  or  both.  In  fact,  DSS  take  many  different 
forms.  They  may  simply  graph  certain  data  as  they  relate  to  other  data,  they  may  contain 


formulae  to  calculate  the  optimal  investment  portfolio  for  a  given  set  of  financial  data  and 
priorities,  or  they  may  help  the  decision  maker  by  allowing  a  particular  simulation  to  be  run 
with  various  combinations  of  input  data.  DSS  are  not  just  being  tested  and  used  by 
computer  companies,  either,  they  are  being  used  by  pharmaceutical  companies,  railroads, 
and  paper  giants  and  even  tested  by  law  enforcement  agencies. 

B.   EXPERT  SYSTEMS 

A  new  variation,  the  Expert  System  (ES),  is  a  computer-based  system  to  emulate  the 
judgment  and  reasoning  of  an  expert.  If  an  expert  could  be  corralled  and  induced  to  tell  his 
secrets,  the  process  of  his  thoughts  could  be  modeled  in  the  form  of  a  rule-based  computer 
program.  Morton  points  out  that  there  is  an  "ES  onus  on  cognitive  processes  underlying  the 
notion  of  expertise."  [Morton  88  p.  17]  In  other  words,  the  ES  tries  to  mimic  the  thought 
patterns  and  processes  through  a  rule-based  set  of  instructions  for  the  computer  to  perform 
given  a  particular  set  of  circumstances.  Although  the  systems  differ  in  that  the  DSS  was 
designed  to  help  the  decision-maker  decide  and  the  ES  was  designed  to  make  the  decision, 
Ford  points  out  that  the  "fundamental  goal  of  DSS  and  ES  is  the  same;  they  seek  to 
improve  the  quality  of  the  decision."  [Ford  85  p.  24]  Mason  goes  on  to  point  out  that"[An 
Expert  System's]  output  is  a  single  course  of  action.  The  decision-maker  retains  just  a 
'veto'  power."   [Mason  69  p.  87] 

Expert  Systems  are  probably  best  described  as  an  evolution  of  DSS.  Some  would 
even  argue  that  an  ES  is  an  intelligent  DSS  [Turbin  &  Watkins  86  p.  139].  Ford  defines 
ES  as  a  "problem-solving  program  that  achieves  good  performance  in  a  specialized  problem 
domain  that  generally  requires  specialized  knowledge  and  skill.  The  systems  process  the 
knowledge  of  experts  and  attempt  to  mimic  their  thinking,  skill,  and  intuition."  [Ford  85 
p.  23] 


Waterman  and  Jenkins  go  on  to  point  out  that  an  expert  system  is  "particularly  useful 
for  problem  domains  that  are  not  well  formalized  and  for  which  no  generally  agreed  upon 
axioms  or  theorems  exist."  [Waterman  &  Jenkins  86  p.  96] 

An  expert  system  is  often  made  up  of  two  components:  a  rule  base  which  contains  the 
"rules  of  thinking"  and  the  inference  engine  which  uses  the  rules  to  infer  a  conclusion. 

1 .  Rule  Base 

The  rule  base  of  an  Expert  System  is  the  set  of  instructions  given  to  the  system  to 
determine  what  action  it  is  to  take  or  what  conclusion  it  is  to  draw  given  a  certain  set  of 
circumstances.  The  rule  base  represents  the  knowledge  of  the  expert,  put  into  a  set  of  "if- 
then"  rules  for  the  system  to  follow  in  a  certain  situation.  The  rule  base  is  the  codification 
of  the  knowledge  that  the  system  is  to  possess.  It  is  the  written  out  form  of  the  expertise 
the  Expert  System  is  to  mimic  when  presented  with  a  certain  problem  or  a  certain  set  of 
problems. 

2 .  Inference  Engine 

The  inference  engine  is  the  heart  of  an  Expert  System,  performing  two  major 
tasks.  According  to  Harmon  and  King,  "it  first  examines  existing  facts  and  rules  [in  the 
rule  base],  and  adds  new  facts  when  possible.  Second,  it  decides  the  order  in  which 
inferences  are  made.  In  doing  so,  the  inference  engine  conducts  the  consultation  with  the 
user."  [Harmon  &  King  85  p.  49]  Sage  points  out  that  the  inference  engine  also  acts  "as 
an  intelligent  interpreter  in  the  selection  of  those  portions  of  the  knowledge  base  which  are 
most  applicable  to  resolution  of  a  given  issue."  [Sage  86  p.  197]  One  of  the  advantages  of 
such  a  setup  is  the  system's  ability  to  generate  an  explanation  of  the  reasoning  process  used 
to  reach  a  conclusion.  The  user  is  then  able  to  validate  the  conclusion  and  possibly  enhance 
his  decision-making  skills  by  following  the  system's  line  of  reasoning. 
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Several  operations  are  involved  with  'firing'  the  rule  base  including  (1)  forward 
and  backward  chaining,  (2)  depth  and  breadth  searches,  and  (3)  monotonic  and  non- 
monotonic reasoning.  In  forward  chaining,  the  inference  engine  takes  existing  data  (facts, 
sensor  inputs,  and  the  like)  and  examines  the  premises  for  rules  to  see  if  they  are  true,  thus 
moving  toward  a  conclusion.  [Harmon  &  King  85]  It  is  best  suited  for  situations  where 
the  solution  needs  to  be  constructed,  usually  from  a  large  number  of  possible  outcomes. 
Backward  chaining,  on  the  other  hand,  starts  with  a  conclusion  (diagnosis,  goal,  etc.)  and 
works  back  through  the  subgoals  to  choose  an  answer.  Backward  chaining  is  commonly 
used  when  the  possible  outcomes  are  known  and  are  reasonably  small  in  number. 

Waterman  and  Jenkins  describe  this  in  terms  of  rules  and  goals.  "The  rules  are 
antecedent  driven.  This  means  that  when  all  the  conditions  of  a  rule  are  true  relative  to  the 
database,  the  rule  'fires'  causing  the  associated  actions  to  be  taken.  The  goals,  however, 
are  consequent  driven.  This  means  that  the  system  is  given  a  condition  to  make  true,  or  in 
effect,  a  question  to  answer  through  deductive  inference.  Here,  the  right  sides  or 
consequents  of  rules  are  examined  to  find  one  that  could  make  the  desired  condition  true. 
When  such  a  rule  is  found,  its  left  side  or  antecedent  is  examined  to  see  if  all  its  conditions 
are  true.  If  they  are,  the  rule  is  fired."  [Waterman  &  Jenkins  86  p.  100] 

A  depth-first  search  of  the  knowledge  base  follows  a  detailed  line  of  reasoning 
concentrating  on  highly  related  facts.  A  breadth-first  search  sweeps  across  all  premises  in  a 
rule  before  digging  for  greater  detail.  [Harmon  &  King  85] 

Monotonic  reasoning  involves  "locking"  previously  validated  premises,  that  is, 
once  the  system  accepts  a  premise  as  true,  it  cannot  reverse  the  logic.  Non-monotonic 
reasoning  is  more  flexible  in  that  the  system  can  reverse  accepted  premises  when  contrary 
information  is  presented.  The  major  drawback  to  such  a  capability  is  that  the  inference 
engine  must  be  able  to  undo  all  premises  based  on  the  invalid  premise. 
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C.   CAN  DSS  AND  ES  WORK  TOGETHER? 

Sage  contends  that  "there  should  exist  many  areas  in  which  the  proper  form  of 
knowledge-based  support  is  a  hybrid  of  an  'expert  system'  and  a  'decision  support 
system.'"  This  is  to  allow  it  to  "support  the  decision  maker  in  the  formulation  or  framing 
of  the  decision  situation  in  the  sense  of  recognizing  needs,  identifying  appropriate 
objectives  by  which  to  measure  successful  resolution  of  an  issue,  and  generating  alternative 
courses  of  action  that  may  resolve  the  needs  and  satisfy  objectives."  [Sage  86  p.  196] 

This  idea  of  the  use  of  DSS  in  concert  with  ES  is  not  a  new  one.  Henderson  finds  that 
a  productive  synergy  can  and  should  exist  between  research  in  the  two  domains 
[Henderson  87].  Turban  and  Watkins  go  on  to  point  out  that  the  combination  of  the  two  in 
an  actual  application  would  not  only  be  able  to  answer  the  question  "what-if?"  but  "will 
also  be  able  to  answer  the  question  'why?'"  [Turban  &  Watkins  86  p.  150]. 

As  the  two  systems  become  intertwined,  it  is  sometimes  difficult  to  differentiate  where 
one  stops  and  the  other  begins.  This  is  not  to  say  that  the  two  are  mutually  exclusive. 
Shane  et  al.  "test"  the  efficacy  of  using  a  decision  support  system  with  an  expert  system 
component."  The  results  of  their  study  "indicate  that  using  Knowledge-Based  DSS  can 
improve  the  decision  maker's  ability  to  identify  and  assess  problems  in  unstructured 
environments."  [Shane  et  al.  86  p.  453] 

We  have  examined  the  concepts  and  uses  of  DSS  and  ES  as  separate  entities.  DSS  fit 
well  into  an  environment  where  the  decision-maker  requires  assistance  in  making  the 
decision,  while  the  ES  replaces  the  decision-maker  and  comes  up  with  the  answer  itself. 
The  environment  of  the  TAO  does  not  call  for  replacement  of  the  TAO,  yet  there  is  a 
tremendous  amount  of  expertise  for  him  to  master,  a  huge  amount  of  information  to  track, 
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and  sometimes  a  very  short  time  in  which  to  make  a  decision.  A  sort  of  hybrid  may  be  in 
order. 

D.    KNOWLEDGE-BASED   DSS 

Based  on  the  intended  use  of  the  TAO  system,  the  term  Knowledge-Based  DSS  is 
preferred  since  the  system  will  be  used  to  provide  an  expert  decision  based  on  its  embedded 
knowledge.  But,  as  will  be  seen,  this  information  is  used  in  an  advisory  role  only.  Goul 
and  Tonge  discuss  the  Knowledge-Based  variation  on  the  DSS  theme.  "Our  perception  is 
that  the  term  incorporates  (1)  the  application  of  DSS  techniques  to  expert-based  system 
construction  and  (2)  expert-based  system  techniques  applied  to  DSS  construction."  [Goul 
and  Tonge  87  p.  451] 

The  systems  apparently  came  into  their  own  not  because  their  assistance  was  so  perfect 
but  because  as  Benbasat  et  al.  point  out  "people  costs  are  greater  than  machine  costs." 
[Benbasat  et  al.  81  p.  753].  The  memory  of  computers  is  also  advantageous  to  the  user  in 
that  once  the  system  is  programmed  to  assist  the  decision  maker  or  to  emulate  the  expert,  it 
can  keep  track  of  a  number  of  variables  without  regard  to  how  many  other  things  are  on  its 
mind  and  how  tense  the  situation  might  have  become  in  the  interim. 

It  is  on  these  characteristics  of  computer  operation  which  this  thesis  will  capitalize: 
their  speed  and  their  ability  to  keep  track  of  a  number  of  simultaneous  complex  activities. 

Numerous  drawings  of  what  such  a  system  would  look  like  attempt  to  point  out  the 
various  interrelationships  among  the  user,  the  computer,  and  the  system  components 
[Harmon  &  King  85,  Sage  86,  Ford  85,  Arcidiacono  88,  and  Turban  &  Watkins  86].  In 
order  to  find  a  basis  to  work  from  on  the  TAO  DSS,  these  drawings  were  examined  to  find 
a  diagram  that  best  fit  the  uses  and  relationships  such  a  system  would  require.  Although 
none  of  these  specifically  fit  the  needs  of  the  TAO  environment  exacdy,  two  were  close  and 
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with  some  combination  and  modification  have  produced  the  generic  diagram  illustrated  in 
Figure  1. 


Justifier 

Knowledge 
Base 

User 
Interface 

Inference 
Engine 

user         ,— 

Blackboard 

Figure  1.  The  Knowledge-Based  DSS 

[Adapted  from  Harmon  &  King  85:  p.  49  and  Turban  &  Watkins  86:  p.  139] 

This  figure  illustrates  the  interaction  of  the  user  with  the  system  through  the  User 
Interface.  The  Justifier,  described  by  Turban  and  Watkins,  is  the  component  that  explains 
to  the  user  the  reasons  for  a  particular  conclusion  (or  interim  conclusion)  by  the  system. 
This  is  the  same  component  referred  to  by  Harmon  and  King  as  the  Explanation 
Subsystem.  The  Blackboard  is  unique  to  the  Turban  and  Watkins  model  and  is  the 
component  "for  recording  intermediate  decisions."  [Turban  &  Watkins  86  p.  140]  The 
Knowledge  Base  contains  factual  rules  to  allow  the  derivation  of  assertions.  As 
Arcidiacono  points  out,  "The  Inference  Engine  uses  assertions  (facts)  and  problem  solving 
expertise  in  the  knowledge  base  to  infer  conclusions.  Its  design  and  function  depends  on 
types  of  knowledge  structures  it  must  process."  [Arcidiacono  88  p.  47]  He  goes  on  to 
point  out  an  important  advantage  of  having  a  separate  Inference  Engine  and  Knowledge 
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Base  since  "a  knowledge  base  for  one  domain  can  be  replaced  by  one  from  another  domain 
without  the  need  for  changes  to  the  inference  engine  or  user  interface."  Although  this  is  a 
deceptively  simple  system,  it  contains  the  necessary  elements  to  describe  the  Knowledge- 
Based  DSS  to  be  considered. 

The  result  of  combining  DSS  and  ES  into  a  Knowledge-Based  DSS  is  what 
Kronenberg  and  Howard  refer  to  as  "judgment  enhancement."  They  further  point  out  that 
"the  development  of  decision  support  capabilities  of  the  character  discussed  here  must  be 
responsive  to  the  unique  style,  priorities,  and  disposition  of  the  manager."  [Kronenberg  & 
Howard  87  p.  52-53]  Huber  also  notes  that  a  manager's  DSS  must  be  compatible  with 
whatever  manual  decision  support  systems  (dss)  he  is  already  using  if  the  system  is  to  be 
used  profitably.  Huber  states  that  "every  manager  has  a  'decision  support  system'  (a  dss,  a 
system  consisting  of  the  information  sources  and  decision  aids  that  the  manager  draws 
upon  as  the  occasion  requires)."  [Huber  81  p.  1]  This  dss  can  take  the  form  of  something 
as  simple  as  a  notepad  and  pencil.  Figure  2  illustrates  the  necessary  relationship  among  the 
decision  maker,  his  dss,  and  his  DSS. 

By  using  the  combined  DSS  and  ES  advantages  in  a  time-constrained,  complex 
environment  of  an  emergency,  the  system  will  accrue  the  advantage  of  a  very  fast  response 
to  the  problem  with  the  additional  plus  that  it  can  keep  track  of  a  number  of  developing 
situations  simultaneously.  This  is  done  with  the  hypothesis  that  these  efforts  can  help  the 
human  TAO  to  make  the  decision  better,  more  responsibly,  with  a  decreased  level  of  stress 
characteristic  of  an  emergency  (discussed  later)  or  of  an  instance  of  information  overload. 
He  is  provided  with  a  much  broader  look  at  all  of  the  possible  alternative  solutions  and 
courses  of  action. 
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Figure  2.  TAO  DSS  must  be  compatible  with  TAO  dss 

E.    INTERFACE 

Additional  areas  to  be  pursued  by  this  research  include  the  type  of  interface  which 
would  best  assist  the  TAO  in  interacting  with  the  system.  As  Hurley  and  Wallace  point 
out,  "Even  an  'optimally'  designed  system  may  go  unused  if  information  is  not  presented 
in  a  form  that  supports  the  decision  situation  and  the  style  of  the  user."  [Hurley  &  Wallace 
87  p.  168]  Several  scholars  have  wrestled  with  this  problem,  and  numerous  interface 
possibilities  now  exist.  For  example,  Liang  has  even  proposed  a  self-adaptive  interface 
that  watches  the  pattern  of  the  man-machine  interaction  and  adjusts  itself  to  meet  the  need. 
[Liang  87]  Shane  et  al.  arrived  at  a  dual  mode  system  which  allowed  guided  and  non- 
guided  sessions.  "The  Non-Guided  Session  is  distinguished  by  the  freedom  provided  the 
decision  maker  to  examine  file  folders  at  will."  They  explain  that  "the  sophisticated 
decision  maker  is  provided  a  more  facilitative  interface."  [Shane  et  al.  86  p.  455] 

Another  facet  to  be  explored  in  the  realm  of  the  interface  is  whether  the  reasoning 
process  followed  by  the  system  to  reach  the  solution  presented  should  be  accessible  to  the 
user.  The  answer  appears  to  be  a  resounding  "yes."  As  Hurley  and  Wallace  found  in  their 
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1987  study,  "When  the  system's  exact  reasoning  process  is  available,  it  is  easier  to 
convince  the  users  that  the  solutions  are  valid  and  reliable."  [Hurley  &  Wallace  87  p.  167] 

F.    CRISES  AND  DSS 

Kronenberg  and  Howard  point  out  that  "sources  of  uncertainty  and  turbulence  are 
magnified  for  those  who  must  deal  with  the  unpredictable  arenas  of  defense,  foreign 
policy,  or  emergency  management."  [Kronenberg  &  Howard  87  p.  49]  They  propose  to 
solve  this  problem  with  a  Decision  Support  System.  They  point  out  that  "Crisis  decision 
making  under  uncertainty  and  stress  has  special  requirements.  In  addition  to  the 
physiological,  skill,  and  experiential  requirements  of  which  we  hear  so  much,  there  are 
extraordinary  information  requirements.  The  senior  manager  is  called  on  to  sort,  validate, 
weigh  alternatives,  and  act  in  an  environment  which  inhibits  all  those  cognitive  processes." 

At  this  point,  we  should  define  "emergency"  in  order  to  clarify  the  focal  point  for  the 
issue.  Elam  and  Isett  point  out  that  crises  are  events  that  have  "no  identical  precedent  on 
which  to  base  a  routine  decision,  and  their  precise  characterization  is  difficult."  [Elam  & 
Isett  87  p.  4]  Crises  have  never  occurred  before  while  emergencies  have  not  only  occurred 
before,  but  there  exist  for  them  contingency  plans  with  which  to  react.  The  reader  may 
consider  this  an  artificial  differentiation,  but  it  is  one  that  is  relied  upon  in  this  research. 

According  to  Smart  and  Vertinsky,  "Designs  for  crisis  decision  making  attempt  to  (1) 
prevent  certain  biases  that  are  specific  to  stressful  situations,  (2)  increase  flexibility  and 
sensitivity  of  line  units,  and  (3)  develop  computational  and  processing  capabilities  in  the 
organization  to  meet  sudden  increasing  demands  imposed  upon  the  decision  units."  [Smart 
&  Vertinsky  77  p.  655].  This  definition  seems  to  suit  the  emergency  situation  of  the  TAO 
by  aiming  at  a  much  better  solution  to  the  problem  facing  the  decision  maker. 
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This  thesis  will  make  use  of  the  phases  of  resolving  a  decision  developed  by 
Govindaraj  et  al.  "1)  uncertainty  reduction  and  2)  threat  classification"  [Govindaraj  et  al. 
85  p.  499]  to  apply  to  the  solution  of  perhaps  the  most  potentially  lethal  of  all  emergencies, 
the  attack  by  a  hostile  military  force  on  our  own  ships. 

Further  research  into  the  area  of  such  systems  for  support  of  decision  making  in  the 
crisis  arena  point  out  that  the  decisions  made  under  the  stress  of  such  a  situation  "are 
subject  to  error  because  the  time  sensitive  nature  of  the  activity  prevents  consideration  of  all 
of  the  viable  alternatives  and  precludes  thorough  investigation  of  the  selected  alternatives." 
[Benoit  et  al.  82  p.  ix].  The  advantages  of  the  use  of  their  system,  developed  for  the  U.  S. 
Air  Force  included  "Thorough  consideration  of  all  relevant  factors  ...,  Prompt  response  to 
crisis  situations,...  [and]  Consistent  behavior."  They  go  on  to  point  out  that  their  system 
"models  the  human  planning  process  by  developing  first  a  set  of  possible  alternatives  and 
then  filling  out  the  details  in  a  series  of  successive  steps."  This  is  the  approach  that  the 
TAO  DSS  will  take. 

Yet  another  study  performed  to  determine  knowledge  requirements  for  Knowledge- 
Based  DSS  points  out  that  "the  decision  maker  starts  with  some  uncertainty  with  regard  to 
the  true  situation,  and  then  looks  for  additional  information  which  may  reduce  this 
uncertainty.  Following  a  cyclic  process  new  information  is  obtained  and  integrated  into  the 
existing  information,  the  situation  is  reassessed  and  if  final  assessment  cannot  be  made, 
further  information  is  requested.  The  process  ends  when  the  decision  maker  decides  that 
he  knows  enough  about  the  situation  to  make  up  his  mind,  or  that  no  additional  sources  of 
information  can  contribute  significantly  (compared  to  their  cost)  to  remove  the  uncertainty 
which  still  remains."  [Ben-Bassat  &  Freedy  83  p.  1].  This  description  seems  to  fit  best 
the  scenario  of  interest:  a  developing  tactical  scenario. 
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Ben-Bassat  and  Freedy  point  out  that  "As  new  findings  come  in,  either  as  a  result  of 
the  decision  maker's  request  or  'voluntarily,'  they  should  be  sorted  with  regard  to  the  entire 
battlefield  structure,  the  triggered  hypotheses  and,  on  the  highest  priority,  with  regard  to 
the  current  goals."  This  assessment  cycle  is  terminated  finally  by  "one  of  the  following 
conditions:  (a)  A  decision  may  be  reached  ...  (b)  Several  triggered  hypotheses  have  not  yet 
been  settled,  however,  the  cost  of  removing  the  remaining  uncertainty  is  relatively  high  ... 
(c)  New  developments  (e.g.,  sudden  enemy  attack)."  These  are  also  valid  points  in  the  at- 
sea  tactical  situation  of  the  TAO. 

Ben-Bassat  and  Freedy  continue  by  saying  that  the  three  key  things  that  they  feel  are 
critical  to  the  success  of  their  system  are  "the  elicitation  and  computer  representation  of  the 
necessary  military  knowledge  for  battle  field  reading  ...  modeling  the  reasoning  and 
inference  processes  which  take  part  during  situation  assessment,  [and]  the  human-machine 
interface."  The  situation,  as  they  aptly  point  out,  "may  basically  be  considered  as  a  puzzle 
building  task." 

Vedder  and  Mason  found  that  for  law  enforcement  puzzles  their  best  results  were  ones 
whose  human-machine  interface  featured  "sharp,  terse  dialogue  based  on  the  scenario 
model  of  a  hostage-taking  incident."  [Vedder  &  Mason  87  p.  405] 

Despite  these  difficulties  and  the  complications  of  solving  the  puzzle,  such  systems 
have  been  built.  Callero  et  al.  produced  a  tactical  air  targeting  aid  in  the  form  of  a  rule- 
based  Expert  System.  They  emphasized  the  need  for  the  structure  of  such  a  system  to 
"permit  rapid  adaptation  within  the  operational  environment."  [Callero  et  al.  86  p.  190] 
They  took  meticulous  care  in  designing  their  system  to  examine  the  needs  of  the  user, 
stating  that  "At  the  outbreak  of  hostilities,  the  officers  cannot  be  expected  to  have  broad 
experience  in  tactical  planning  ....  clearly  [there  is]  a  need  for  sophisticated,  automated  aids 
to  help  regularize  the  process  and  assist  the  targeteer  in  making  the  best  possible  selection." 
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Klahr  et  al.  produced  TWIRL  (Tactical  Warfare  in  the  ROSS  Language),  a  ground 
combat  simulation.  Their  study  is  particularly  interesting  for  its  extensive  discussion  of  the 
system's  object-oriented  framework.  In  this  setup,  they  explain,  "Objects  are  organized  in 
a  hierarchy  of  class-subclass  links  [and]  This  hierarchy  allows  an  object  to  inherit 
automatically  the  attributes  and  behaviors  of  the  classes  to  which  it  belongs."  They  point 
out  that  the  "organization  of  knowledge  around  objects  facilitates  modularity,  modifiability, 
and  maintenance  of  the  simulation"  and  go  on  to  assert  that  "most  behaviors  are  associated 
with  classes  rather  than  instances.  For  our  purposes,  we  are  more  concerned  with  the 
ability  to  represent  various  types  of  military  units  than  we  are  with  the  ability  to  represent 
large  quantities  of  any  particular  type  of  unit."  [Klahr  et  al.  86  p.  231]  This,  too,  will 
become  an  important  concept  in  the  design  of  a  DSS  for  TAOs. 

G.  SUMMARY 

DSS  are  designed  to  assist  decision-makers  in  doing  their  jobs,  while  ES  are  intended 
to  replace  the  decision-maker  altogether  by  automating  the  process  of  making  the  choice.  A 
marriage  between  the  two  results  in  a  Knowledge-Based  DSS,  one  capable  of  assisting  the 
decision-maker  through  its  internal  database  of  expertise.  This  is  certainly  a  field  that 
deserves  further  examination,  and  a  number  of  applications  have  already  been  built  to  allow 
these  new  fields  to  become  available  in  the  crisis  decision-making  environment.  They  can 
help  by  unemotionally  dispensing  the  expertise  in  time  of  crisis,  keeping  track  of  myriad 
information  for  the  decision-maker,  and  speeding  up  the  decision  process.  It  appears  to  be 
a  logical  choice  for  a  TAO  decision  aid,  but  such  a  conclusion  cannot  be  positively  stated 
until  more  specific  TAO  requirements  are  characterized. 
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III.      DSS  APPLICATIONS  FOR  THE  MILITARY. 


A.  BACKGROUND 

The  idea  of  DSS  for  military  use  is  not  a  new  one.  In  1983,  Vice  Admiral  Samuel 
Gravely,  USN  (Retired)  wrote  about  the  Integrated  Information  Display  (IID)  System 
which  was  designed  to  assist  the  Fleet  Commanders-In-Chief  (CINC)  handle  the 
voluminous  amounts  of  information  to  "be  collected,  correlated,  analyzed,  and 
disseminated."  [Gravely  83  p.  53]  De  Balogh  points  out  that  the  military,  in  fact, 
developed  the  concept  of  "artificial  intelligence  -  artificial  instinct"  systems,  which  are 
"sensor-based  to  monitor  conditions  within  a  complex  environment  and  communicate  a 
warning  plus  initiate  immediate  societal  response  within  seconds."  [De  Balogh  85  p.  101] 
He  also  discusses  requirements  for  such  a  system  which  would  include  "A  combination  of 
natural  language  and  icon-based  interactive  dialog  environment,  a  continuous,  systematic 
updating  effort  of  major  proportions"  to  keep  up  with  the  dynamic  environment  [De  Balogh 
85  p.  86]  The  necessity  for  continuous,  systematic  update  also  exists  in  the  TAO 
environment,  where  the  burgeoning  array  of  missiles  available  is  astonishing,  and  where 
the  parameters  of  each  are  changeable  to  keep  the  enemy  off  guard. 

It  seems  strange  that  such  a  helpful  tool  has  not  been  provided  to  the  shipboard  TAO 
who  deals  with  much  of  the  same  data  and  has  many  of  the  very  same  information 
requirements  as  the  CINC's  Operations  Duty  Officer.  Furthermore,  the  TAO  is  really 
where  the  rubber  meets  the  road  —  he's  the  one  controlling  the  employment  of  missiles  and 
other  weapons  against  enemy  forces.  His  need  is,  if  anything,  more  critical  than  that  of  the 
shore-based  CINCs'. 
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There  is  a  danger  here,  too  in  that  the  system  provided  to  a  TAO  might  not  fit  his 
needs.  Sage  contends  that  "Perhaps  the  most  damning  charges  of  all  that  affect  potential 
user  willingness  to  use  the  system  are  the  feeling  that  it  significantly  interferes  with  the 
'normal'  way  of  thinking  about  problems,  or  that  it  cannot  adapt  to  changes  in  problem 
specifications,  or  that  it  does  not  produce  intermediate  results  of  value,  or  that  it  does  not 
really  address  the  actual  problems  that  exist."  [Sage  86  p.  202]  The  importance  of  design 
is  evident  here  ;  it  is  possible  to  actually  make  the  job  harder  if  the  decision  aid  does  not 
meet  the  requirements. 

Goul  and  Tonge  point  out  that  critical  to  the  design  of  any  such  system  "are  the  needs 
and  desires  of  the  system  user  as  they  influence  design."  [Goul  &  Tonge  87  p.  450]  They 
state  that  several  of  the  approaches  used  today  completely  ignore  these  crucial  elements. 

Callero  et  al.  point  out  that  "Human  decision  making  is  inherently  unstructured"  and 
suggest  "that  automated  aids  specifically  designed  to  reflect  the  human  decision  process  can 
contribute  to  better  judgments."  To  do  this,  they  favor  the  "knowledge-engineering 
problem-solving  approach  in  which  human  domain  knowledge  is  essential,  and  judgment, 
experience,  and  intuition  play  a  larger  role  than  mathematical  algorithms  and  stochastic 
formalisms."  [Callero  et  al.  86  p.  186] 

Weiss  suggests  that  "perhaps  the  most  powerful  reason  for  the  success  of  the  Order  of 
Battle  Version  1  Knowledge  Base  [OB  1KB],  a  U.  S.  Army  battlefield  advisor,  was  that  the 
end  user  played  a  key  role  in  scoping,  designing,  and  implementing  the  system."  [Weiss  86 
p.  93]  She  also  points  out  that  "Because  of  the  inherent  transparency  of  the  logic  of  the 
system  operations  and  the  ease  of  access  to  information,  OB  1KB  was  used  in  the  field  as 
an  accepted,  powerful  aid  in  the  decision  making  process.  More  importantly,  in  order  to 
use  OB  1KB,  the  soldiers  did  not  have  to  change  the  way  in  which  they  did  their  job." 
[Weiss  86  p.  95] 
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Thus,  the  voice  of  Huber  echoes  its  warning  that  in  order  to  be  used  and  useful,  the 
DSS  must  be  compatible  with  the  manager's  dss.  The  system  for  a  TAO  will  have  to  be 
flexible  and  adaptive  to  his  changing  environment.  Waterman  and  Jenkins  suggest  that  the 
user  and  the  system  may  actually  feed  off  of  each  other,  improving  each  other's  decision 
skills.  "On  the  basis  of  rules  provided  by  the  analysts,  the  system  will  reach  conclusions 
with  which  the  analysts  may  disagree.  In  this  case,  the  analysts  will  be  compelled  to 
reexamine  the  basis  for  their  deductions,  and  this  may  lead  to  the  formulation  of  new  rules 
for  the  system."  [Waterman  &  Jenkins  86  p.  97] 

This  iterative  learning  should  also  be  applied  to  the  design  of  the  system's  logic. 
Courbon  et  al.  describe  this  as  a  "methodology  based  on  the  progressive  design  of  a  DSS, 
going  through  multiple  as-short-as-possible  cycles,  in  which  successive  versions  of  the 
systems  under  construction  are  utilized  by  the  end-user."  (cited  in  [Morton  88  p.  14]) 
Morton  distinguishes  this  approach  from  prototyping  "because  the  initial  system  could 
actually  become  the  real  system,  not  just  a  pilot  test."  [Morton  88  p.  1 1] 

To  design  a  system  that  allows  continuous  update,  that  adapts  to  changes  in  the 
problem,  that  supports  rather  than  interferes  with  the  "normal"  way  of  doing  things,  that 
produces  intermediate  results  of  value,  and  that  is  compatible  with  the  TAO's  dss  is  a  tall 
order.  It  requires  a  more  thorough  understanding  of  data  available  to  the  TAO,  information 
required  by  him,  and  the  jobs  of  the  assistants  who  provide  this  information. 

1.     TAO's  Principal  Assistants 

The  TAO  himself  has  a  number  of  assistants  to  help  him  carry  out  his  duties. 
They  do  this  by  keeping  him  informed  of  the  various  data  which  become  available  aboard 
the  ship  -  data  which  might  help  to  narrow  down  the  identity  of  the  contact.  These 
assistants  range  from  the  Officer  of  the  Deck  (OOD)  on  the  bridge  of  the  ship  to  the 
Electronic  Warfare  specialist  (EW).  These  inputs  are  illustrated  in  Figure  3. 
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Figure  3.  Information  Inflow  to  the  TAO 

The  Officer  of  the  Deck  is  in  charge  of  maneuvering  the  ship  but  his  position 
allows  him  to  see  a  number  of  things  otherwise  unavailable  to  the  TAO. 

The  lookout  has  an  even  better  view  since  he  is  frequently  placed  higher  on  the 
ship,  thus  extending  his  visual  range.  The  lookout  is  often  not  experienced  enough  to 
know  what  he  is  looking  at,  but  his  raw  data  can  still  be  helpful  to  those  who  can  interpret 
what  he  reports. 

The  Signalmen  on  the  signal  bridge  can  send  a  flashing  light  message  to  other 
ships  to  ask  them  for  their  identities.  This  can  be  valuable  even  with  Soviet  ships  since  a 
standard  communication  procedure  accepted  by  both  sides  allows  for  at  least  rudimentary 
communication  to  indicate  one's  intentions  to  avoid  collisions.  Signalmen  are  also  trained 
in  the  art  of  visually  identifying  other  men-of-war,  as  well  as  merchant  ships  and  pleasure 
craft. 
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The  Operations  Specialists  (OS)  in  the  ship's  Combat  Information  Center 
provide  the  radar  tracking  of  contacts  to  determine  their  course  and  speed,  closest  point  of 
approach,  etc.  This  can  be  valuable  in  narrowing  the  field  if  a  particular  contact  is  observed 
proceeding  at  a  speed  faster  than  a  class  of  ship  is  believed  capable  of  or  has  a  radar  return 
indicating  a  larger  aircraft  than  a  hostile  fighter. 

The  Sonar  Technicians  (ST)  provide  tracking  on  sonar  and  can  similarly  provide 
course/speed  data  and  the  narrowing  of  contacts'  identities  as  well  as  certain  tonal  data 
which  emanate  from  contacts.  These  "tonals"  come  from  main  feed  pumps,  reduction 
gears,  etc.  and  are  sometimes  distinctive  enough  to  identify  a  particular  class  of  contact. 
Propeller  blade  rates  also  can  provide  identity  data  through  speed  of  rotation  and  the  actual 
count  of  the  number  of  blades. 

The  Electronic  Warfare  specialists  can  passively  listen  to  the  electronic  signals 
emanating  from  a  particular  platform  and  determine  from  the  combination  of  signals  what 
emitters  a  platform  has  and  therefore  what  that  platform  might  be.  This  guess  is  based  on 
known  data  of  which  classes  of  ships,  submarine,  and  aircraft  carry  that  particular  suite  of 
emitters.  Some  of  the  EW's  gear  is  automated  and  will  determine  from  the  pulse  repetition 
frequency,  jitter,  pulse  rate,  etc.  what  the  platform  is  while  other  less  automated  gear  will 
merely  present  the  parameters  listed  above  to  the  ES  and  allow  him  to  identify  it  from  this 
data. 

2 .      Further  Complications 

The  potential  danger  of  all  of  the  supposedly  helpful  assistance  the  TAO  is 
receiving  above  is  that  he  is  constandy  interrupted  by  the  various  people  who  are  informing 
him  of  some  new  tidbit  of  information  to  add  to  his  growing  collection.  These 
interruptions  cause  his  analytic  train-of-thought  to  be  broken  and  for  information  to  be  lost, 
forgotten,  or  worse  yet,  to  be  attributed  to  the  wrong  contact,  actually  worsening  and 
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further  confusing  the  tactical  picture.  It  is  possible  that  better  decisions  would  result  from  a 
tactical  team  consisting  of  the  TAO  and  a  TAO  DSS  to  keep  track  of  correlations  and  to  help 
deduce  answers.  This  relationship  is  illustrated  in  Figure  4. 

This  confusion  coupled  with  the  TAO's  responsibility  to  brief  the  Commanding 
Officer  on  the  tactical  situation,  to  watch  the  Officer  of  the  Deck's  maneuvers  to  ensure  the 
ship  is  in  safe  waters,  to  ensure  the  ship's  weapons  systems  are  unmasked,  to  manage  the 
CIC  team,  and  his  responsibility  to  run  exercises  can  further  exacerbate  the  problem  of  the 
TAO's  information  glut  and  can  cause  information  to  never  be  correlated  or  used. 

Even  in  a  calm,  ideal  environment  with  all  information  available  and  all  data 
flows  perfect,  there  is  a  problem  beyond  that  of  merely  correctly  identifying  neighboring 
contacts. 

3  .      Determining  Hostile  Intent 

The  environment  of  the  TAO  is  further  complicated  by  the  fact  that  even  when  he 
knows  the  identity  of  the  various  contacts,  he  is  still  unsure  of  their  intentions.  They  may 
mean  his  ship  harm  or  they  may  just  be  projecting  their  presence  on  the  high  seas  and 
monitoring  the  U.  S.  ships. 

To  help  the  TAO  through  this  dilemma,  there  exists  a  set  of  criteria  which,  when 
they  are  met,  may  determine  that  the  contact  has  a  so-called  "hostile  intent." 

These  criteria  include  such  things  as  having  the  missile  doors  on  a  contact  open 
(as  viewed  by  a  surveillance  aircraft  or  a  member  of  the  ship's  company)  or  radiating  a 
friendly  ship  with  a  fire  control  radar.  The  hostile  intent  may  be  directed  at  other  friendly 
forces  (not  the  TAO's  own  ship),  such  as  a  friendly  helicopter. 
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Figure  4.  The  TAO/  DSS  Team 

In  these  situations,  a  DSS  could  help  by  asking  the  TAO  questions  relative  to  a 
particular  contact's  status  at  the  time  to  determine  whether  hostile  intent  is  present  in  his 
actions.  Yet  another  complication  which  comes  into  play  is  that  the  Rules  of  Engagement 
for  a  particular  situation  may  be  entirely  different  from  those  for  the  same  situation  in 
another  ocean.  These  rules  are  created  and  invoked  by  the  various  fleet  commanders  under 
the  direction  of  the  National  Command  Authorities  (the  President  and  the  Secretary  of 
Defense).  Those  of  the  Persian  Gulf  are  different  from  those  of  the  Mediterranean  because 
of  known  hostile  intent  in  the  Gulf.  Here  again  the  DSS  could  help  the  situation  by 
knowing  those  rules  and  presenting  the  appropriate  actions  for  the  TAO  to  carry  out  for  a 
given  situation. 

But,  all  of  this  is  just  conjecture  if  not  backed  up  by  interviews  with  real  TAOs  to 
determine  what  would  be  the  most  compatible  with  their  collective  dss,  what  would  assist 
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the  "normal"  way  of  examining  problems,  what  information  is  most  needed  by  the  TAO 
and  when,  and  what  type  of  interface  would  be  easiest,  most  effective,  and  fastest  to  use. 

B  .   DESIGN  OF  THE  RESEARCH  APPROACH 

In  order  to  solicit  this  information  effectively,  careful  design  went  into  the  specific 
research  approach.  Since  most  TAOs  are  not  familiar  with  DSS  in  general  or  with  the  more 
specific  aspects  which  this  research  was  attempting  to  reach  into,  it  was  decided  that  a 
questionnaire  was  likely  to  confuse  the  respondents  more  than  it  would  benefit  the 
research.  In  light  of  this,  the  value  of  having  a  live  interviewer  present  to  explain  what  he 
was  trying  to  get  at  was  considered  worth  the  extra  scheduling  problems  and  the  extra  time 
involved  in  conducting  the  interviews.  This  coupled  with  the  tendency  for  more 
information  and  insight  to  come  out  during  a  spoken  interview  than  in  the  environment  of  a 
respondent  having  to  write  this  information  down  on  a  questionnaire  clinched  the  decision. 

The  classic  work  of  Sprague  and  Carlson  describes  four  major  areas  for  consideration 
in  DSS  design  —  Representations,  Operations,  Memory  Aids,  and  Control  Mechanisms 
(ROMC).  These  four  areas  guided  the  design  of  the  interview  structure  for  elicitation  of 
information  from  the  future  users  —  the  TAOs  being  interviewed.  [Sprague  &  Carlson  82] 

1 .      Representations 

This  element  of  the  DSS  is  the  way  in  which  the  information  gleaned  from  the 
input  data  is  presented  to  the  TAO.  Sprague  and  Carlson  assert  that,  "Any  activity  in  a 
decision-making  environment  takes  place  in  the  context  of  some  conceptualization," 
[Sprague  &  Carlson  82  p.  102]  and  it  is  the  representations  which  facilitate  this 
conceptualization.  The  effort  here  was  to  extract  from  the  TAOs  interviewed  the  best  way 
to  show  the  information  needed.  This  part  of  the  interview  revolved  around  the  graphics, 
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text,  and  video  presentations.  This  is  important  as  noted  in  the  previous  chapter  because,  if 
presented  poorly,  the  information  may  not  be  used  at  all. 

This  issue  was  approached  through  the  use  of  a  mockup  screen  made  prior  to  the 
interview  and  shown  to  the  user  for  his  critique  and  suggestions.  This  screen  simulation  is 
reproduced  in  Figure  5. 
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Figure  5.  Sample  Screen  To  Determine  Proper  Method  For  Representation 

2.      Operations 

The  Operations  component  involves  the  performance  of  the  software  and  indeed 
the  system  as  a  whole  from  the  user's  perspective.  Included  here  are  the  various 
capabilities  of  the  system  which  the  decision-maker  can  use.  He  could,  for  instance,  use 
the  system  to  ascertain  the  identity  of  the  contact  and  also  a  recommended  course  of  action 
to  deal  with  a  certain  contact.  This  aspect  of  the  DSS  is  critical  because  if  the  information  is 
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derived  too  slowly  or  the  information  gleaned  is  not  the  information  needed,  the  system 
again  will  not  be  useful. 

Some  of  this  information  was  elicited  from  the  TAOs  through  the  use  of  a 
hypothetical  tradeoff  graph  (reproduced  here  as  Figure  6)  illustrating  the  possible  degree  of 
certainty  that  the  decision  is  correct,  juxtaposed  with  the  amount  of  time  required  to  reach 
that  level  of  confidence. 
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Figure  6.  Time/Certainty  Tradeoff 

3 .      Memory  Aids 

This  aspect  of  the  DSS  involves  the  method  used  to  remind  the  TAO  of  the 
identity  of  a  contact  and  other  pertinent  information.  Sprague  and  Carlson  point  out  that  a 
decision  can  often  be  represented  in  terms  of  its  presentation.  [Sprague  &  Carlson  82] 
Here,  the  idea  is  to  find  the  fastest,  most  effective  method  to  remind  the  TAO  with  the  least 
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least  likelihood  he  might  misinterpret  what  he  is  seeing.  This  could  involve  a  change  in 
symbology,  a  special  way  to  flag  a  particular  contact,  text  presentations,  or  audible  signals. 
One  of  the  questions  asked  to  determine  the  optimum  method  to  remind  the  TAO 
was,  "Should  the  information  gleaned  by  the  system  through  processing  the  input  data  be 
automatically  presented  to  the  TAO  every  so  often;  Or  should  the  information  determined 
by  the  system  be  presented  to  the  TAO  only  when  he  queries  the  system  for  it?" 

Answers  to  this  question  would  reveal  an  appropriate  storage  format  as  well  as 
important  "operations"  capabilities  of  the  system. 
4 .      Control  Mechanisms 

Usually  consisting  of  option  menus  or  another  form  of  user-system  dialog,  this 
DSS  component  is  aimed  at  finding  the  best  way  for  the  TAO  to  direct  the  efforts  of  his 
computer  assistant.  Sprague  and  Carlson  state  that  this  element  of  the  paradigm  is 
"intended  to  help  decision  makers  use  representations,  operations,  and  memory  to 
synthesize  a  decision-making  process  based  on  their  individual  styles,  skills,  and 
knowledge."  [Sprague  &  Carlson  82  p.  107]  It  can  take  the  form  of  a  keyboard,  a  voice 
control  module,  a  mouse  (or  track  ball,  discussed  later),  a  light  pen,  or  a  touch  screen. 
Here,  ease  of  use  and  effectiveness  of  operation  are  key. 

Goul  and  Tonge  used  this  paradigm  and  allowed  the  user  to  choose  within  the 
limits  of  some  of  his  Representations,  Operations,  Memory  Aids,  and  Control 
Mechanisms.  They  state  that  "This  has  been  shown  to  be  effective  in  enhancing  the 
expert's  understanding  of  and  direct  involvement  with  the  inference  engine  used  in  the 
system."  [Goul  &  Tonge  87  p.  465] 

One  of  the  questions  asked  to  discover  the  optimal  method  to  be  used  for  TAO 
control  of  the  system  was,  "How  often  should  the  TAO  be  queried  for  new  information? 
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Should  he  be  queried  at  all  or  should  he  simply  be  able  to  enter  information  in  the  order  he 
receives  it  and  WHEN  he  receives  it?" 

This  ROMC  paradigm  was  used  to  develop  the  structure  for  the  TAO  interviews 
to  be  conducted  (Appendix  A).  In  fact,  the  interview  structure  was  organized  along  the 
lines  of  the  ROMC  paradigm  in  order  to  channel  the  interviewees'  thoughts  into  one  area  at 
a  time. 

There  are  areas  where  the  distinct  components  become  blurred,  however.  Where 
possible,  these  areas  were  used  to  guide  the  interviewees  to  the  next  topic.  In  some  cases, 
this  transition  was  avoided  to  permit  a  more  separate  examination  of  the  topics  at  hand. 
Representations,  for  instance,  was  not  joined  with  Control  Mechanisms  despite  the 
common  link  between  them  -  the  user- system  interface. 

In  other  cases,  much  stronger  links  between  areas  were  evident  in  interviews 
than  had  been  anticipated.  The  Operations  issue  led  more  than  a  few  interviewees  into  a 
discussion  of  Control  Mechanisms. 

Although  this  was  set  up  as  a  structured  interview  process,  these  links  were 
pursued  rather  than  suppressed.  It  is,  after  all,  the  interviewees  who  know  best  what  they 
want,  and  this  freedom  allowed  several  TAOs  to  generate  new  possibilities  for  the  design 
of  the  system.  These  ideas  are  discussed  in  Chapter  Four. 

C.   INTERVIEW  DYNAMICS 

The  interviewees  chosen  were  picked  at  random  from  among  the  Surface  Warfare 
Officers  in  the  student  population  of  the  Naval  Postgraduate  School  with  a  preference  for 
the  more  experienced  officers.  This  thesis  will  therefore  use  the  intuitive  reasoning  that 
more  experience  will  give  a  better  reading  as  to  the  value  of  a  DSS  in  the  TAO's 
environment.  It  will  work  from  a  starting  assumption  that  the  more  information  derived  for 
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a  TAO  from  the  data  available  to  him,  the  better  the  tactical  picture  he  will  have.  And  the 
better  the  tactical  picture,  the  better  off  he  will  be  when  the  shooting  war  actually  starts. 

Interviews  lasted  from  30  minutes  to  100  minutes  depending  on  how  much  the 
interviewees  had  to  say.  The  more  experienced  officers  consistently  had  the  longer 
interviews,  and  the  two  most  senior  interviewed  both  said  that  it  is  long  overdue  to  have  a 
system  out  there  to  help  the  TAO.  Several  officers  got  quite  animated,  and  their  excitement 
led  to  several  of  the  ideas  presented  in  Chapter  4.  Not  all  interviews  were  conducted  in 
complete  privacy,  and  in  three  non-private  interviews,  Naval  officer  passersby  added 
constructive  comments  to  what  they  had  overheard.  Two  of  these  comments  proved 
beneficial  in  the  evaluation  of  user-system  interface  possibilities. 

D.   SUMMARY 

This  is  not  the  first  attempt  to  assist  military  decision-makers  through  the  use  of  DSS, 
but  the  TAO  environment  lacks  any  such  help.  That  environment  consists  of  numerous 
complications,  arising  from  the  complexity  of  determining  identity  and  subsequently  hostile 
intent.  Although  provided  with  assistants  to  help  in  this  process,  the  TAO  is  still 
confronted  with  a  difficult  problem. 

In  the  design  of  the  interview  structure,  the  Representations,  Operations,  Memory 
Aids,  and  Control  Mechanisms  paradigm  was  used  to  organize  the  interviews  roughly 
along  the  lines  of  the  design  of  an  actual  system.  The  interviews  themselves  generated  a 
good  deal  of  enthusiasm  and  valuable  input  for  the  research.  The  results  of  the  interviews 
and  the  resulting  design  of  the  system  are  presented  in  Chapter  IV. 
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IV.      EVALUATION  OF  THE  EFFICACY  OF  HOTDAM!  (HART'S 
OPERATIONAL  TACTICAL  DECISION  AIDING  MICROCOMPUTER) 

FOR  THE  TAO  ENVIRONMENT 

Based  on  the  TAO  interviews  conducted,  the  appropriateness  of  a  Knowledge-Based 
DSS  for  the  problem  presented  by  the  environment  of  the  TAO  was  analyzed.  Most  TAOs 
questioned  felt  that  such  a  system  would  be  of  significant  benefit  to  them  in  the 
environment  of  the  developing  tactical  situation  for  use  in  analytic  form  to: 

•  figure  out  what  contacts  are  and 

•  remind  the  TAO  what  the  team  opinion  regarding  the  identity  of  the  contact  is  and 

•  recommend  courses  of  action. 

Since  these  were  in  fact  the  experts  for  whom  the  system  was  being  designed,  the 
"users,"  the  assumptions  made  by  the  author  about  the  need  for  such  a  system  appear  to 
have  been  accurate.  A  more  specific  breakdown  of  the  interview  results  follows. 

A.   DEMOGRAPHICS 

A  total  of  fourteen  TAOs  were  interviewed.  Of  those,  two  had  prior  enlisted  service, 
and  one  is  a  reservist.  All  have  stood  watch  as  TAO,  and  all  are  U.  S.  naval  officers. 
Seven  officers  were  Lieutenants,  four  were  Lieutenant  Commanders,  and  three  were 
Commanders.  Two  of  the  Commanders  have  already  had  command  at  sea. 

The  average  years  of  commissioned  service  for  the  group  was  10.32  (ranging  from  a 
low  of  4.5  years  to  a  high  of  23  years).  The  average  length  of  time  since  qualified  as  a 
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Surface  Warfare  Officer  (SWO)  was  7.8  years  (with  the  low  here  being  one  year,  and  the 
high  being  15).  The  true  average  may  actually  be  higher  than  reported  since  formal  SWN 
qualification  was  actually  implemented  only  15  years  ago. 

Levels  of  education  ranged  from  bachelors  degrees  to  "almost  Ph.D."  The  major  areas 
of  study  included:  Ocean  Engineering  (1),  Analytical  Management  (2),  Information 
Systems  (2),  Marine  Science  (1),  Nuclear  Physics  (1),  Finance  (1),  English  (1),  History 
(1),  Electrical  Engineering  (1),  Communications  (1),  Economics  (2),  Political  Science  (2), 
Operations  Analysis  (1),  Computer  Science  (1),  Mechanical  Engineering  (1),  and 
Oceanography  (1). 

Most  interviewees  characterized  themselves  as  able  to  navigate  around  on  computers 
pretty  well  (8),  one  listed  his  experience  as  extensive,  three  listed  experience  as  minimal, 
and  two  claimed  experience  on  one  specific  program  only. 

B .   INPUTS  FOR  THE  TAO  DSS 

Despite  the  wide  diversity  in  level  of  experience  on  computers,  years  of  experience  as 
a  TAO,  educational  background  and  majors,  and  the  wide  variety  of  ship  types  on  which 
the  interviewees  had  served,  there  was  a  general  agreement  about  the  need  for  a  DSS  to 
assist  TAOs,  the  willingness  to  use  such  a  system,  and  the  importance  of  speed  and  of  user 
interface. 

All  of  the  TAOs  interviewed  expressed  a  willingness  to  use  a  system  to  help  narrow 
the  identity  of  contacts  closing  their  ships  with  the  few  (but  almost  unanimous)  stipulations 
that  it: 

•  work  fast 

•  work  effectively 
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•  work  efficiently 

•  have  a  friendly  interface  and 

•  be  highly  reliable. 

Many  of  the  specified  characteristics  of  the  system  that  the  TAOs  interviewed  would  be 
comfortable  with  deserve  a  more  thorough  discussion  in  order  to  make  it  clear  just  why 
they  are  necessary.  In  fact,  these  descriptions  will  later  serve  as  the  design  criteria  of  such 
a  system. 

1 .  User   Friendliness 

Most  said  they  thought  other  TAOs  would  use  it  too  if  it  were  user-friendly,  if 
they  had  had  computer  experience,  if  the  system  were  simple  to  understand,  if  they  were 
"year  group  '65  or  younger",  "if  they  had  tested  the  system  first  in  a  non-volatile 
environment,"  and  if  the  interface  were  unintimidating.  One  respondent  would  say  only 
that  it  is  "hard  to  say." 

2.  Usefulness 

Asked  about  how  useful  the  TAO  DSS  would  be,  interviewees  answered 
everything  from,  "It  would  be  an  extra  layer  of  unnecessary  complexity"  to  that  it  would  be 
"very  much  of  a  help,"  with  most  responses  indicating  that  helpfulness  depends  on  speed 
of  response  (6  TAOs  specified  this)  or  reliability  (4  specified). 

A  breakdown  of  the  ranking  by  importance  to  the  TAO  of  information  and 
services  able  to  be  provided  by  the  system  is  given  in  Appendix  B. 

3 .  Modularity 

The  consensus  seemed  to  be  that  the  system  should  be  modularized  to  allow  the 
viewing  of  only  a  particular  class  of  target  (i.e.,  air,  surface,  or  subsurface)  but  that  it 
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should  be  switchable  to  other  categories  or  to  all  categories  with  a  single  action  (such  as  a 
menu  choice). 

4 .  Chaining 

All  TAOs  agreed  that  the  system  should  also  be  able  to  work  backwards  from  an 
educated  guess  (i.e.,  an  intelligence  report  that  a  certain  contact  is  in  the  area)  until  it  hits 
something  that  refutes  the  evidence  or  until  the  TAO  is  satisfied  that  the  contact  is  correctly 
identified. 

5 .  Speed  Versus  Accuracy 

Five  TAOs  preferred  the  system  give  them  a  fast  answer  rather  than  pursuing 
every  possibility  and  coming  up  with  the  most  accurate  answer  possible.  Four  preferred 
the  most  accurate  answer  only,  and  four  preferred  to  receive  the  fastest  answer  first 
followed  by  a  more  accurate  answer  later.  One  suggested  that  both  options  be  available, 
selectable  by  menu  depending  on  the  situation.  This  option  seems  a  good  choice  since  the 
opinions  here  were  so  split. 

Shown  the  tradeoff  graph  of  time  versus  certainty  of  identity  shown  as  Figure  7, 
most  declared  a  separate  preference  for  the  time/certainty  pairing  for  air  targets  than  for 
surface/subsurface  targets  due  to  the  greater  speed  at  which  air  scenarios  develop.  This 
means  that  the  inference  time  required  for  the  air  contacts  must  be  reduced  or  that  inference 
time  for  both  air  and  surface  tracks  must  be  improved.  Half  of  the  interviewees  declared 
the  choices  on  the  scale  unsatisfactory  and  stated  that  the  performance  represented  was  just 
too  slow.  One  pointed  out  that  this  again  is  a  situation-dependent  performance  issue.  The 
remainder  of  the  results  are  presented  in  Appendix  B. 

6 .  Non-Monotonic  Versus  Monotonic  Reasoning 

All  interviewees  indicated  a  desire  to  be  allowed  to  change  elements  of  data  that 
had  already  been  entered  without  being  required  to  start  the  entire  scenario  over. 
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Figure  7.  Time/Certainty  Tradeoff 

7 .      Platform   Specific 

Eleven  respondents  felt  that  it  would  be  valuable  to  make  the  system  platform- 
specific  in  order  to  exploit  various  sources  of  information  that  are  available  but  frequently 
not  used  by  a  ship  (i.e.,  size  of  a  target  extrapolated  from  detection  range  from  one's  own 
ship  on  a  particular  radar).  The  remaining  three  felt  that  this  feature  would  add 
unnecessarily  to  the  cost  of  a  "generic"  system.  This  may  be  a  valid  point  in  view  of 
funding  cuts  already  going  into  effect  for  the  military.  However,  the  value  of  the 
information  attainable  from  exploiting  the  data  already  available  on  each  class  of  ship  is 
potentially  tremendous.  Such  data  might  have  prevented  the  downing  of  Iran  Air  flight 
655,  had  the  USS  Vincennes  been  able  to  determine  the  size  of  that  aircraft.  In  what  has 
been  called  "one  of  the  Navy's  worst  nightmares  come  true"  [Matthews  88  p.  1],  perhaps 
better  interpretation  of  the  available  data  and  an  unemotional  evaluation  of  the  situation 
(without  the  stress  of  the  current  gunboat  battle  in  progress)  could  have  prevented  the 
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misidentification  and  engagement.  A  retired  Navy  Captain  has  pointed  out  that  "With  the 
information  he  had  in  his  hand  and  the  time  limit  he  was  under,  he  did  the  right  thing." 
[Matthews  88  p.  4]  It  may  be  possible  someday  to  provide  the  TAO  and  CO  of  the  ships 
we  send  into  harm's  way  a  better  capability  for  the  evaluation  of  the  information  they  have 
to  permit  a  better  decision  in  a  situation  where  his  ship  is  "in  imminent  danger."  [Matthews 
88  p.  18]  This  could  result  in  better  defense  of  our  ships  while  avoiding  the  tragic  results 
of  an  incorrect  interpretation  of  data. 

8 .  Optimal  Number  of  Tracks 

As  for  the  optimal  number  of  contacts  for  the  TAO/DSS  team  to  handle  together, 
the  answers  varied  widely  based  on  the  past  experience  of  the  TAOs  of  the  maximum 
number  of  contacts  they  had  had  to  handle.  Answers  ranged  from  5  to  256,  but  a  realistic 
approach  would  be  to  not  limit  the  number  of  contacts  at  all  except  due  to  hardware 
constraint  or  to  the  detriment  of  the  actual  software  performance.  The  more  the  computer 
can  keep  track  of,  the  less  the  TAO  has  to  worry  with.  An  absolute  minimum  should  be 
around  100  tracks. 

9 .  Man-Machine  Interaction 

Interviewees  were  about  evenly  split  between  preferences  for  a  man-machine 
dialog  featuring  voice  interaction  (7)  and  one  featuring  a  mouse  or  track  ball  (6).  Some 
TAOs  expressed  a  preference  for  more  than  one  interface  mode,  resulting  in  votes  for 
keyboard  interface  (2),  light  pen  (1),  touch  screen  (1),  and  function  keys  (1),  as  well.  One 
TAO  expressed  a  strong  dislike  for  keyboard  use,  and  one  an  extreme  distaste  for  voice 
interface.  The  mouse  or  track  ball  is  probably  the  best  suited  to  the  environment  and 
functions  to  be  performed  by  the  TAO. 

Pull-down  menus,  windows,  and  icons  were  preferred  12  to  1  over  an  MS-DOS 
command  line  format  with  one  abstainer  saying,  "Only  testing  will  tell"  and  the  other  that 
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this  was  not  applicable  because  of  his  preference  for  natural  language  interface.  One 
answer  was  particularly  telling,  "MS-DOS  would  kill  most  TAOs,"  indicating  that  these 
users  would  probably  find  that  environment  so  user-hostile  that  the  system  would  go 
unused  ~  an  unacceptable  situation. 

10.  Learning  the  System 

Most  interviewees  expressed  a  preference  for  having  a  live  instructor  to 
demonstrate  the  system  come  to  each  ship  with  the  system  (1 1  to  3),  and  most  preferred  a 
short  instruction  manual  with  examples  (8)  to  a  detailed  one  (2)  or  a  short  set  of 
instructions  with  more  detailed  explanations  in  the  back  (4).  All  but  one  TAO  thought 
instruction  on  such  a  system  should  be  given  at  all  TAO  training  facilities  ashore,  and  10  of 
those  13  thought  hands-on  training  in  laboratories  was  necessary,  while  the  others  felt  a 
brief  introduction  would  be  sufficient. 

11.  Willingness  to  Use 

If  this  system  showed  up  on  their  ships,  ten  TAOs  said  they  would  take  the  time 
to  learn  to  use  it  (the  other  4  would  be  too  busy),  but  eight  interviewees  said  TAOs  would 
become  too  dependent  on  such  a  system;  the  remaining  six  recognize  a  potential  problem  in 
that  area  but  feel  that  drills  and  the  TAO  mentality  would  guard  against  this.  These  drills  do 
tend  to  focus  on  just  such  failures,  and  most  TAOs  have  been  burned  often  enough  to 
know  that  if  such  a  system  can  fail,  it  will  at  the  worst  possible  time.  Thus,  too  much 
dependence  on  the  system  will  probably  be  avoided. 

All  interviewees  felt  such  a  system  would  be  tremendously  helpful  by  replacing 
weapon  system  parameter  lookups  and  platform  weapons  list  lookups  in  Naval  Warfare 
Publications  (NWP),  even  if  it  fulfilled  no  other  purpose. 
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12.  Chauffeur 

Twelve  TAOs  preferred  to  have  a  chauffeur  to  operate  the  system  to  minimize 
distraction  of  the  TAO.  One  TAO  suggested  using  Junior  Officers  (JO  --  usually  O-l  and 
0-2)  as  chauffeurs;  this  would  accomplish  the  same  purpose  with  the  additional  advantage 
of  training  the  J.O.s.  Eleven  interviewees  said  Electronic  Warfare  specialists  (EWs)  were 
an  inappropriate  choice  for  chauffeur,  while  three  felt  they  were  a  good  choice  if  watched 
carefully.  EWs  are  usually  placed  somewhere  out  of  sight,  frequently  behind  a  curtain 
making  the  "watch  carefully"  option  impracticable,  but  the  fact  that  this  was  brought  up 
illuminates  another  pertinent  consideration.  EWs  have  a  reputation  for  being  less  than 
completely  alert  due  to  the  fact  that  they  are  seated,  their  job  in  most  cases  requires  attention 
only  occasionally,  and  even  then  much  of  their  gear  is  already  so  automated  that  no  thought 
is  required.  If  this  reputation  is  founded,  EWs  are  probably  a  poor  choice. 

13.  Indication  of  Certainty 

Since  the  system's  degree  of  certainty  is  an  important  consideration  to  the  TAO 
in  a  situation  of  contact  identity,  this  should  be  presented  to  him.  That  is,  it  is  crucial  to  the 
TAO  to  know  how  sure  the  system  is  of  the  identification  displayed  to  him.  If  the  system 
were  only  10%  certain,  it  would  be  unwise  to  rely  solely  on  that  identification.  Normally 
referred  to  as  a  Confidence  Factor,  this  degree  of  certainty  can  be  presented  in  a  variety  of 
ways.  Exactly  half  preferred  a  numerical  percentage  be  presented  to  them,  and  half 
preferred  a  graphical  display  such  as  a  thermometer  graph.  A  combination  of  the  two  could 
be  achieved. 

Most  TAOs  (12)  favored  a  TAO-initiated  data  input  to  the  system  with  a  system 
query  only  if  15  minutes  or  more  pass  with  no  update  to  a  particular  contact.  The  others 
preferred  no  system  queries  at  all!  It  would  seem  prudent  to  have  the  system  prompt  the 
TAO  after  several  minutes  to  remind  him  of  the  contact's  existence,  even  if  most  input  was 
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initiated  by  the  TAO  or  chauffeur.  This  could  help  prevent  simple  forgetfulness  from 
becoming  part  of  a  potentially  disastrous  tactical  scenario.  Several  suggested  integrating 
the  system  into  the  ship's  existing  combat  system,  but  one  Lieutenant  Commander  made 
the  valid  and  important  point  that  this  would  eliminate  the  benefit  of  redundancy. 

Most  TAOs  also  expressed  a  need  to  be  able  to  see  why  the  system  arrived  at  a 
particular  answer  —  what  the  trace  of  its  logic  was.  This  would  improve  confidence  in  the 
system  and  allow  the  TAO  to  pinpoint  where  it  is  that  he  disagrees. 

C.   DESIGN  OF  HOTDAM! 

The  TAOs  interviewed  provided  valuable  input  on  required  system  performance,  user- 
system  interface,  services  to  be  provided,  modularization,  methods  of  logic  chaining, 
method  of  database  search,  the  necessity  to  change  data  already  entered,  training 
capabilities,  user's  manual,  and  a  need  to  review  the  logic  trace  the  system  followed  to 
reach  a  conclusion.  Based  on  this  data  and  on  the  rigors  and  space  constraints  of  the  at-sea 
tactical  environment,  the  HOTDAM!  (Hart's  Operational  Tactical  Decision  Aiding 
Microcomputer!)  system  was  designed  to  best  fit  all  of  these  requirements. 

1 .      Hardware 

Mini-  or  Microcomputer-Based  Personal  computers  have  evolved  into  highly 
sophisticated  systems  capable  of  handling  the  rigors  of  Knowledge-Based  applications. 
They  are  rugged,  light-weight,  inexpensive  and  readily  available,  allowing  adaptation 
without  the  need  for  shipboard  modification.  The  new  generation  of  microprocessors  (Intel 
80386  and  Motorola  68030)  provide  the  power  and  speed  needed  to  handle  a  Knowledge- 
Based  DSS  implementation.  Should  the  memory  requirements  exceed  the  capacity  of 
personal  microcomputers  (even  with  added  memory),  the  system  could  be  implemented  on 
minicomputers  with  little  added  space  requirements.  The  system  could  eventually  be  run 


42 


on  some  of  the  Navy's  existing  combat  systems  computers  if  the  system  is  ultimately  tied 
into  the  system.  Since  the  microcomputer  choice  is  preferred,  this  setup  is  illustrated  as  sit- 
down  consoles  in  Figure  8. 

a.     Keyboard  Data  Input  and  Mouse  Control 

In  the  dynamic  environment  of  the  Combat  Information  Center  (CIC),  the 
TAO  needs  to  be  free  from  traditional  keyboard  data  entry  for  speed  of  input  and  to 
concentrate  on  the  varied  information  presented.  A  voice  recognition  interface  with  large 
video  display  would  alleviate  the  need  to  be  seated  at  a  computer  terminal,  providing 
improved  flexibility.  But,  voice  interface  (at  this  writing)  is  frequently  anything  but  user- 
friendly  and  highly  unreliable. 
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Figure  8.  HOTDAM!  in  Action 

Additionally,  the  TAO  environment  is  probably  an  inappropriate  place  to  use 
such  an  interface  since  the  oft-experienced  tension  can  change  the  TAO/chauffeur's  voice 
enough  to  cause  it  to  be  not  recognized  by  the  system  -  a  very  undesirable  result.  Hence, 
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keyboard  entry  by  a  chauffeur  is  the  much  more  practical  (and  safer)  way  to  go.  A  mouse 
or  track  ball  should  be  used  for  the  TAO's  control  of  and  interface  with  the  system  due  to 
its  speed  and  almost  intuitive  operation.  A  chauffeur  would  be  recommended  to  free  the 
TAO  from  the  distraction  of  data  input,  but  this  should  not  be  required  since  some  TAOs 
really  prefer  to  do  it  themselves  (if  they  have  time). 

b.  Cursor  Manipulation 

Much  research  has  been  done  to  determine  the  best  and  most  efficient  way 
to  interact  with  a  computer.  Lohr  even  conducted  specific  work  to  determine  the  best 
interface  for  the  tactical  system.  [Lohr  80]  He  favors  an  upside-down  mouse  or  "track 
ball"  because  "downward  force  from  the  palm  is  required  in  addition  to  angular  force, 
making  this  a  very  stable  'grip'  generally  immune  to  the  effects  of  a  moving  platform," 
such  as  the  pitching  and  rolling  surface  ship  of  the  TAO's  environment.  He  also  discusses 
other  interface  methods  and  points  out  that  there  is  an  advantage  to  the  use  of  light  pens 
also  because  of  their  "familiar  pointing  action  ('...  want  to  pick  that  one')."  But,  in  the 
final  analysis,  he  says,  "there  is  no  single  positioning  device  that  is  ideal  for  all 
applications,  thus  a  general  purpose  tactical  control  system  should  be  available  with  a 
variety  of  control  devices."  [Lohr  80  p.  26]  For  two  reasons  the  choice  for  HOTDAM!  is 
the  track  ball.  First,  it  is  already  familiar  to  most  TAOs  since  is  is  in  current  use  aboard 
many  ships  for  console  manipulation.  Second,  the  track  ball  has  already  proven 
satisfactory  for  shipboard  use  due  to  its  general  immunity  to  the  inherent  pitfalls  of  that 
environment. 

c.  High  Resolution   Color  Graphics  Display 

As  Benbasat  et  al.  found  in  their  gaming  studies,  graphics  are  generally 
more  beneficial  to  the  decision  maker  than  text.  [Benbasat  et  al.  81]  To  provide  the  "big 
picture"  the  system  should  display  the  tactical  scenario  much  like  the  Naval  Tactical  Data 
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System  (NTDS)  with  standard  symbology.  This  pictorial  summary  allows  the  TAO  to 
rapidly  digest  the  situation  in  concert  with  system-generated  recommendations  without  the 
need  to  learn  new  symbology.  A  flashing  symbol  could  be  used  to  attract  attention  should  a 
hostile  contact  be  identified,  for  example,  and  the  color  of  contacts  should  be  changed  to 
indicate  their  identities  (friendly,  unknown,  hostile).  Choice  of  color  here  is  important.  As 
Lohr  points  out,  "Blue  is  six  times  less  noticeable  than  green  and  three  times  less  noticeable 
than  red."  [Lohr  80  p.  27]  So,  if  the  goal  is  to  have  someone  take  particular  note  of  a 
contact,  green  is  the  (almost  counter-intuitive)  color  to  display  it  in.  It  is  important  to  note 
that  making  a  contact  more  conspicuous  can  cause  the  dangerous  side  effect  of  confusion 
particularly  if  something  is  presented  in  a  counter-intuitive  color  to  that  in  which  it  is 
usually  represented.  Should  this  confusion  turn  out  to  be  the  result,  the  colors  used  should 
be  the  intuitive  ones  or  the  color  issue  might  be  abandoned  altogether.  Lohr's  evaluation, 
however,  was  made  with  light  contacts  on  a  dark  background  and  may  not  apply  to  the 
same  colors  with  a  light  background.  A  further  intuitive  point  to  be  made  here  is  that  a 
flashing  contact  would  be  more  noticeable  than  any  others.  The  real  evaluation  of  efficacy 
should  be  tested  on  real  TAOs  in  a  close  simulation  of  a  true  Combat  Information  Center  in 
a  tactical  situation. 
2.     Software 

The  Knowledge-Based  DSS  is  a  hybrid  creature,  a  blend  of  the  best  parts  of  both 
the  Expert  System  and  the  DSS.  As  discussed  in  Chapter  2,  there  are  many  views  about 
how  exactly  the  various  parts  of  such  a  system  should  interact  to  best  assist  the  decision 
maker.  The  diagram  presented  in  the  discussion  there  was  a  blend  from  two  other 
diagrams  in  an  attempt  to  best  present  the  system  in  light  of  the  TAO's  environment. 

As  the  design  of  the  system  progressed  after  the  interviews  were  complete,  a 
number  of  alterations  to  the  generic  idea  (and  the  diagram  presented  in  Chapter  2)  of  the 
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Knowledge-Based  DSS  became  necessary  in  order  to  incorporate  the  unanticipated  needs 
of  the  TAO.  Among  these  changes  were  the  addition  of  a  chauffeur  and  his  associated 
keyboard  interface,  a  relocation  of  the  Justifier  to  a  position  immediately  between  the  TAO 
interface  and  the  Inference  Engine,  and  the  further  specification  that  the  TAO  interface  be 
one  using  Track  Ball  Manipulation  of  the  cursor.  This  new  design  is  presented  in  graphic 
form  as  Figure  9. 
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Figure  9.  HOTDAM!,  the  TAO  DSS 

The  Chauffeur  is  a  weak  link  in  this  system  —  his  input  will  only  be  as  fast  as  his 
typing  speed,  and  his  accuracy  will  only  be  as  good  as  his  understanding  of  the  input  data 
and  the  direction  he  receives  from  the  TAO. 

While  it  is  recognized  that  this  represents  a  quantum  leap  in  the  cost  of  such  a 
system,  it  would  be  hard  to  place  a  value  on  the  increased  speed  with  which  the  system 
could  provide  an  evaluation  in  a  situation  of  an  approaching,  possibly  hostile  contact  in 
time  of  doubt,  confusion,  and  information  overload.  This  assistance  to  the  TAO  could 
prove  invaluable  in  the  evaluation  of  the  contact,  thus  preventing  the  engagement  of  a 
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neutral  contact  while  ensuring  the  proper  actions  are  taken  against  a  contact  showing  hostile 
intent. 

The  more  specific  software  requirements  of  the  HOTDAM!  system  are  discussed 
in  detail  below  and  represented  graphically  in  Figure  10. 
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Figure  10.  Proposed  Design  of  HOTDAM!,  the  TAO  DSS 

a.     Menu  Driven  with  Extensive  Help  Function 

The  TAO  needs  direct,  explicit  dialog  with  the  DSS  for  speed  and  accuracy 
of  interaction.  As  Vedder  and  Mason  experienced,  to  be  successful,  "  It  should  be  noted 
that  should  the  funds  be  available  to  build  one,  an  ideal  system  would  be  connected  into  the 
various  sensors  of  the  ship  and  to  the  existing  combat  system  in  order  to  significantly 
improve  the  input  speed  and  accuracy  of  the  data.  A  system  used  in  a  crisis  must  provide  a 
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terse,  direct  dialog.  [Vedder  87]  The  TAO  needs  such  dialog  for  speed  and  accuracy  of 
interaction  with  the  system.  This  is  accomplished  with  multiple  windows  and  icons  where 
appropriate.  An  extensive  HELP  function  provides  a  built-in  tutor  to  guide  the  TAO-in- 
training. 

b.  Novice  and  Expert  Modes 

The  inexperienced  TAO  needs  guidance  to  adequately  train  on  the  ES;  the 
veteran  needs  a  streamlined  system  that  enhances  system  operation  by  alleviating 
extraneous  interface  dialog.  The  dual-mode  approach  (achieved  through  a  simple  menu 
selection  by  the  TAO)  satisfies  this  requirement  by  giving  the  novice  extensive  on-line 
instruction  while  the  expert  is  given  minimal  dialog  for  efficient  system  operation.  The 
novice  mode  could  be  used  not  only  to  train  TAOs  in  system  use  but  also  to  train  J.O.s  in 
important  tactical  thinking. 

c.  Multiple  Threat  Analysis 

As  with  the  AEGIS  weapon  system,  analysis  and  tracking  of  multiple 
targets  is  required  to  adequately  fight  the  ship.  Since  the  tactical  DSS  is  not  integrated  with 
ship  sensors  and  weapon  systems,  the  program  will  be  limited  only  by  the  memory 
constraints  of  the  hardware.  This  is  in  order  to  improve  the  human  information  processing 
limitation  of  "seven  (plus  or  minus  two)",  espoused  by  Miller.  [Miller  56  p.  52]  Since  the 
TAO  can  keep  track  of  about  seven  contacts  by  himself,  the  system  will  improve  his 
capacity  by  remembering  them  for  him  and  by  reminding  him  of  them  through  visual  cues 
and  changing  symbols.  Miller  found  that  recoding  can  increase  this  capacity  by  allowing 
groupings  to  assist  in  this  remembering  process.  In  fact,  the  system  could  be  used  to 
recode  for  the  TAO  through  such  cues  as  symbol  shape  to  allow  better  TAO  processing. 
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d.  Rules  of  Engagement  (ROE)  Integration 

ROE  are  guidelines  promulgated  by  higher  authority  that  spell  out  under 
what  circumstances  we  can  legally  engage  a  hostile  adversary  (e.g.,  return  fire  only  when 
fired  upon).  To  properly  advise  the  TAO,  the  ROE  in  effect  must  be  integrated  into  the 
inference  engine.  These  rules  can  be  modified,  as  needed,  by  trained  personnel  as  the  ship 
changes  operating  areas  (i.e.,  Mediterranean,  Atlantic,  Pacific,  Persian  Gulf,  etc.). 

e.  English-like   Syntax 

A  fourth  generation  programming  language  (e.g.,  ROSIE  from  RAND)  may 
be  used  to  provide  DSS  code  which  promotes  ease  of  software  maintenance  and  user 
understanding. 

/.     Modularized   Construction 

By  using  a  modular  concept,  the  system  software  is  capable  of  addition  and 
deletion  of  platform-specific  applications  (i.e.,  surface  warfare,  airborne  anti-submarine 
warfare,  and  subsurface  warfare).  This  provides  a  standard  shell  with  rule-base  modules 
for  various  environments.  Modularized  construction  also  has  the  advantages  that 
"technological  advances  can  be  easily  accommodated"  and  that  it  "greatly  aids  in  the  ability 
to  repair  a  system."  [Lohr  80  p.  28] 

g.     Platform-Specific  Data  Base  and  Inference  Engine 

Each  class  of  surface  combatant  has  a  unique  suite  of  sensors  and  weapons 
systems.  The  DSS  data  and  rule  bases  will  be  tailored  to  the  specific  class  of  ship  to 
correctly  infer  weapons  employment  recommendations  (e.g.,  weapons  range  versus  target 
range).  This  will  include  data  on  electronic  signal  emission,  acoustics,  radar  signatures, 
etc. 
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h.     Dual  Mode  Chaining 

Because  tactical  situations  normally  require  construction  of  the  conclusion 
from  an  extensive  database  of  threats  the  system  will  use  forward  chaining.  In  situations 
where  the  TAO  wishes  to  evaluate  a  non-system  assessment  (i.e.,  opinion  of  an 
experienced  TAO  or  Intelligence  Officer),  backward  chaining  can  be  invoked,  adding 
flexibility  to  the  system.  This  could  be  accomplished  by  a  single  menu  selection  for  each 
contact.  Several  systems  exist  today  that  have  this  capability. 
i.      Breadth-First  Search 

With  the  myriad  threats  facing  our  surface  forces,  error  tolerance  in  target 
analysis  must  be  extremely  low.  In  support  of  this  requirement,  a  breadth-first  search  will 
be  used  to  consider  all  possible  matches  against  data  input. 
j.      Non-Monotonic   Reasoning 

The  time-critical  aspect  of  the  seaborne  tactical  environment  necessitates  the 
ability  to  reverse  incorrect  assumptions.  Use  of  non-monotonic  reasoning  in  the  inference 
engine  will  allow  for  this,  eliminating  the  need  to  re-enter  the  entire  data  set  when  an  error 
is  discovered. 

k.     Man-Machine  Interaction 

In  discussing  the  interface  between  a  tactical  system  and  its  user,  Lohr 
points  out  that  "The  interface  must  be  easy  to  use,  fast,  and  efficient."  [Lohr  80  p.  25] 
Vice  Admiral  Gravely  pointed  out  that  flexibility  is  a  certain  advantage  to  a  system  like  this. 
To  make  the  system  fit  Huber's  criterion  of  compatibility  with  the  user's  dss,  the  Integrated 
Information  Display  system,  discussed  in  Chapter  3,  can  "be  modified  to  reflect  the 
operating  style  of  an  individual  commander."  [Gravely  83  p.  60].  Weiss  points  out  the 
importance  of  "minimizing  the  amount  of  typing  required,"  a  system  that  "suggests  the 
kinds  of  answers  required"  through  a  set  of  sample  responses,  the  use  of  "standard 
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symbology,"  the  transparency  of  system  logic,"  and  "the  ease  of  access  to  information." 
[Weiss  86  p.  95] 

As  just  discussed,  the  system  will  use  a  breadth-first  search  to  cover  all 
probable  target  classifications.  After  preliminary  administrative  tasks  (time,  date,  position, 
system  mode,  and  target  identification  code  update)  the  user  is  first  presented  with  a  menu 
prompting  entry  of  target  characteristic  based  on  method  of  detection  (e.g.,  Electronic 
Support  Measures  (ESM),  radar,  sonar,  etc.). 

One  should  be  cautioned  that  system  output  may  not  result  in  a  definitive 
target  identification.  Even  after  exhaustive  database  searches,  it  is  possible  that  several 
platforms  will  satisfy  input  target  characteristics.  The  result  is  a  listing  of  possible  target 
identifications  for  TAO  evaluation. 

3 .      Is  it  Possible  to  Implement  Such  a  System  Today? 

Though  numerous  so-called  Expert  System  Shells  are  on  the  market,  many  are 
appropriate  for  only  "toy"  problems.  An  Expert  System  Shell  is  a  software  package  that 
allows  "expertise"  to  be  put  into  it  and  used  without  the  extra  time  and  difficulties 
associated  with  building  the  entire  system  from  scratch.  Most  such  systems  allow  the 
simple  insertion  of  the  rule  base  and  inference  strategies.  Examination  of  those  Expert 
System  Shells  known  to  exist  narrowed  the  field  to  seventeen  possibilities  for  the 
implementation  of  HOTDAM!. 

Several  of  the  companies  approached  expressed  concern  about  corporate  spies 
and  were  hesitant  to  discuss  their  systems  with  this  researcher,  citing  "proprietary 
information."  This  was  rather  a  surprising  response  since  the  questions  asked  were  not 
those  involving  specific  programming  techniques  but  ones  about  whether  their  systems 
allow  both  forward  and  backward-chaining,  whether  or  not  they  use  non-monotonic 
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reasoning,  etc.  Hence,  some  of  the  information  about  these  systems  had  to  be  obtained 
through  espionage  —  in  technical  magazines! 

The  seventeen  systems  which  made  it  through  the  initial  screening  and  were 
further  researched  are  presented  in  Appendix  C,  along  with  the  list  of  those  five  systems 
which  made  it  through  the  second  screening  process  as  finalists.  The  addresses  of  the 
corporate  makers  of  these  five  finalists  are  also  included  in  Appendix  C. 

Personal  Consultant  +  was  noteworthy  for  its  ease  of  portability  to  other  types  of 
computers  including  mainframes,  LISP  workstations,  IBM  AT  and  compatibles,  T  I 
Explorer  Workstations,  IBM  PS/2  Models  50,  60,  and  80,  and  Compac  Deskpro  386.  It 
has  an  easily  understood  presentation  of  confidence  factors,  advanced  graphics,  and  a  good 
explanation  facility.  However,  it  had  serious  shortcomings  in  that  it  is  not  particularly 
friendly  to  users  not  already  familiar  with  Expert  Systems  and  seems  to  suffer  from  chronic 
out-of-memory  errors  when  dealing  with  large  databases  of  information  (such  as  will  be 
required  by  HOTDAM!). 

Goldworks  can  brag  of  its  ease  of  use,  its  explanation  facility,  its  object 
orientation,  and  its  certainty  factors.  But,  it  suffers  from  a  lack  of  ability  to  perform 
knowledge-based  reasoning  without  continual  referral  to  DBase  III4"  files  and  seems  better 
suited  to  knowledge  acquisition  and  more  flexible  inferencing  than  the  TAO's  environment 
presents. 

An  overall  winner  was  found  in  NEXPERT,  which  not  only  provides  the  non- 
monotonic reasoning,  rule-based  approach,  and  both  forward  and  backward  chaining  but 
also  allows  for  direct  input  from  outside  sensors  should  the  HOTDAM!  system  ever  be 
expanded  to  incorporate  direct  connection  to  the  ship's  combat  system.  Its  object-oriented 
format  fits  nicely  with  the  structure  of  the  data  to  be  included  in  the  HOTDAM!  system,  and 
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the  multiple  inheritance  capability  will  allow  for  the  easier  structuring  of  the  knowledge 
base. 

Automatic  goal  generation  will  allow  the  system  to  pursue  such  goals  as 
"Civilian  Airliner,"  and  when  that  goal  is  determined  to  be  false  to  generate  and  pursue 
another  follow-on  goal.  Its  in-depth  explanatory  function  is  available  at  any  point  during  a 
session.  A  "browsing"  mechanism  will  allow  a  review  of  the  data  and  would  facilitate 
training. 

Although  the  system  runs  on  IBM  AT  or  AT  compatibles  under  Microsoft 
Windows  operating  environment  or  on  the  MicroVAX,  its  graphics  and  user  interface  are 
quite  similar  to  those  of  the  Macintosh  (and  to  the  display  represented  in  Figure  5). 

A  series  of  editors  allows  modification  of  rules,  objects,  classes,  and  properties 
should  the  occasion  arise.  This  could  facilitate  the  easy  incorporation  of  changes  to 
friendly  or  enemy  weapons  systems  and  to  the  Rules  of  Engagement. 

Hence,  it  appears  that  the  HOTDAM!  system  can  be  implemented  through  the 
existing  Expert  System  Shell  called  NEXPERT.  This  will  allow  for  an  easier  and  faster 
resulting  system,  and  provides  for  the  further  expansion  of  the  system  to  include  sensor 
input,  if  and  when  that  enhancement  comes  to  pass. 

D.   SUMMARY 

This  chapter  has  discussed  the  demographic  data  on  the  TAOs  interviewed  and  has 
examined  their  inputs  for  the  characteristics  of  HOTDAM!.  It  then  explored  the  design  of 
the  system  from  hardware,  software,  inference  engine,  and  interface  points  of  view, 
discussing  the  benefits  of  the  various  choices  made. 

In  order  to  create  a  system  that  is  useful  and  usable  for  a  TAO,  it  was  necessary  to 
look  at  the  way  a  TAO  makes  decisions  and  whatever  dss  he  uses  to  ensure  compatibility 
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and  therefore  usability.  It  is  necessary  to  have  a  system  rugged  enough  to  stand  up  to 
shipboard  life,  a  system  with  an  optimal  data  input  mode  and  an  easy  to  use  control 
mechanism. 

The  system  must  account  for  novices  as  well  as  experts  and  to  allow  analysis  of 
multiple  threats  simultaneously.  Inferences  must  be  made  in  all  the  ways  needed  by  a 
TAO,  and  the  resulting  recommendations  must  be  in  accordance  with  the  effective  Rules  of 
Engagement.  In  short,  the  system  must  be  designed  to  best  fit  the  TAO's  unique 
environment  and  information  needs. 

The  resulting  design  of  HOTDAM!  optimizes  the  mix  of  the  numerous  criteria 
necessary  to  have  the  best  Knowledge-Based  DSS  possible  for  the  TAO.  The  design 
complete,  further  research  was  conducted  to  determine  whether  any  Expert  System  shell  on 
the  market  today  could  meet  the  requirements  posed  by  HOTDAM!.  Happily,  there  were 
enough  qualifiers  to  permit  a  rather  choosey  comparison  and  selection  of  the  extremely 
capable  NEXPERT. 
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V.   SUMMARY,  CONCLUSIONS,  AND  RECOMMENDATIONS 

A.   SUMMARY 

This  thesis  has  examined  the  fields  of  DSS,  ES,  Knowledge-Based  DSS,  and  the 
previous  efforts  at  crisis  and  emergency  support  by  these  two  fields  of  study.  It  then 
examined  the  environment  of  the  U.  S.  Navy's  shipboard  Tactical  Action  Officer  to 
determine  whether  the  TAO  could  be  helped  by  DSS,  ES,  or  Knowledge-Based  DSS. 

That  environment  is  a  complex  one  of  multiple  contacts  closing  the  force,  competing 
tasks,  possible  information  overload,  electronic  warfare,  smart  weapons,  and  an  enemy 
who  is  trying  his  best  to  deceive  the  TAO  about  his  real  identity  and  intentions.  All  of  these 
complications  must  be  taken  into  account,  all  possibilities  must  be  explored,  all  options 
examined,  and  careful  evaluation  of  the  action  to  be  taken  made,  all  in  an  environment 
which  inhibits  these  things.  It  is  this  environment  that  may  be  assisted  by  a 
microcomputer-based  Decision  Support  System  to  keep  track  of  some  of  this  data  for  the 
TAO,  to  extract  valuable  information  from  the  data,  and  to  recommend  courses  of  action  to 
the  TAO  based  on  its  evaluation  of  the  situation. 

In  order  to  determine  whether  this  would  be  a  practical  and  realistic  help  to  the  TAO, 
an  interview  structure  was  developed  to  guide  interviews  of  real  TAOs  to  determine 
whether  this  was  an  appropriate  solution  to  the  TAO  problem.  The  structure  for  the 
interviews  was  based  on  the  Representations,  Operations,  Memory  Aids,  and  Control 
Mechanisms  paradigm  of  Sprague  and  Carlson  in  order  to  channel  the  discussion  into  these 
vital  aspects  of  an  effective  decision  aid.  Fourteen  TAOs  of  varying  length  of  experience, 
time  of  qualification,  and  operational  experience  were  interviewed  to  determine  whether 
such  a  system  would  be  of  perceived  value  to  the  TAO  faced  with  identifying  unknown, 
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potentially  hostile  contacts.  The  respondents  identified  the  most  needed  assists  the  system 
could  provide,  the  required  information  to  be  deduced,  the  necessary  speed  of  operation, 
and  the  best  interface  method. 

Based  on  the  interviews  conducted,  the  overwhelming  view  appears  to  be  that  a 
Knowledge-Based  DSS  would  be  a  great  help  to  TAOs.  But  there  was  a  great  deal  of 
disagreement  about  exacdy  what  help  would  be  most  beneficial  to  the  TAO. 

The  small  sample  size  may  not  have  allowed  enough  screening  to  get  a  flavor  for  the 
majority  of  TAOs.  However,  the  information  gathered  here  does  provide  a  valuable  set  of 
needs  since  those  interviewed  are  quite  experienced  in  the  area  of  TAO  operations.  A 
follow-up  set  of  interviews  could  be  conducted  with  TAOs  currently  serving  at  sea  and 
standing  watch  in  the  position  on  a  day-to-day  basis.  These  TAOs  may  come  up  with 
further  valuable  suggestions  to  improve  the  system  discussed  here. 

Although  more  research  may  be  required  to  accomplish  the  implementation  of  the 
system,  this  thesis  establishes  a  good  basis  from  which  to  work  and  some  design  criteria 
which  appear  relatively  certain  based  on  the  general  agreement  about  their  inclusion.  The 
following  criteria  must  be  included: 

•  Micro-  or  Minicomputer-Based 

•  Keyboard  Data  Input  by  a  Chauffeur 

•  Track  Ball  Control  by  the  TAO 

•  High  Resolution  Color  Graphics  Display 

•  Menu-Driven  with  Extensive  Help  Function 

•  Novice  and  Expert  Modes 

•  Multiple  Threat  Analysis 

•  Rules  of  Engagement  (ROE)  Integration 
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•  English-Like  Syntax  in  the  programming  language 

•  Allow  easy  modification  to  ROE 

•  Modularized  Construction 

•  Platform- Specific  Rule  Base  and  Inference  Engine 

•  Dual  Mode  (Forward  and  Backward)  Chaining 

•  Breadth-First  Search 

•  Non-Monotonic  Reasoning 

•  As  User-Friendly  an  Interface  as  Possible. 

B.   CONCLUSIONS 

Perhaps  as  important  a  result  of  this  thesis  as  the  specification  of  these  criteria,  though, 
is  the  determination  from  real  TAOs  that  they  unanimously  felt  that  such  a  system  would  be 
a  significant  help  to  the  TAO  in  resolving  unidentified  contacts  in  a  non-combat 
environment.  Knowledge-Based  DSS  show  a  great  deal  of  promise  for  application  to  the 
TAO  environment.  Isett  found  that  his  mockup  DSS  significantly  helped  to  improve  the 
decision  quality  of  the  Air  Force's  equivalent  of  the  TAO  in  crisis  situations.  [Isett  87] 
Results  of  experiments  involving  Knowledge-Based  DSS  do  not  always  show  that  they 
help,  however.  Goul  et  al.  found  "no  significant  difference  between  the  control  group  and 
the  group  using  the  DSS  with  a  complete  knowledge  base"  in  their  study  of  strategic 
planning.  [Goul  et  al.  86  p.  82]  The  environment  of  the  developing  tactical  scenario  is 
neither  that  of  crisis  nor  that  of  true  strategic  planning  and  will  probably  require  something 
of  a  different  approach. 
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C.   RECOMMENDATIONS 

Because  TAOs  will  not  always  be  able  to  explain  what  they  need  prior  to 
implementation,  a  prototyping  solution  tested  in-the-flesh  by  fleet  TAOs  is  likely  to  be  the 
most  successful  method  of  implementation.  An  iterative  development  allows  the  system  to 
be  honed  to  the  razor's  edge  required  by  the  TAOs  of  today's  Navy.  This  prototype  could 
be  developed  by  students  for  trials  in  fleet  exercises  or  inport  drills  in  order  to  get  a  feel  for 
how  such  a  system  would  be  evaluated  under  realistic  conditions  and  show  how  such  a 
system  could  be  made  better.  Such  a  system  could  be  developed  using  the  Expert  System 
shell  NEXPERT,  as  determined  in  the  final  stage  of  this  research.  Naval  Ocean  Systems 
Command  (NOSC)  could  then  assume  the  watch  in  order  to  fully  examine  the  work 
completed,  make  its  evaluation  and  improvements,  if  any,  and  proceed  with 
implementation. 

A  danger  to  be  avoided  in  the  implementation  of  HOTDAM!  is  the  tendency  of  any 
bureaucracy  to  sometimes  decide  for  a  user  what  it  is  that  the  user  needs  and  how  best  to 
provide  the  services  required.  In  order  to  ensure  the  use  and  usefulness  of  a  system  like 
this,  it  is  mandatory  that  the  TAOs  be  consulted  for  their  views  at  each  step  of  the  process 
leading  to  completion. 

A  terrible  waste  of  money,  effort,  and  lives  could  result  from  TAOs  not  using  this 
system  simply  due  to  the  fact  that  it  does  not  provide  what  they  really  need.  The  only  way 
to  achieve  this  is  through  iterative,  adaptive  development.  Iterative  development,  or 
prototyping,  is  supported  in  a  number  of  the  packages  available  for  the  development  of 
Expert  Systems  and  is  specifically  marketed  for  the  Personal  Consultant  +,  Goldworks, 
and  NEXPERT.  A  possible  further  assurance  against  developing  a  system  the  TAO  will 
not  use  would  be  to  develop  prototypes  using  each  of  these  systems  and  allow  TAOs  to 
choose  the  one  that  most  closely  fits  their  needs.  In  doing  this,  though,  care  must  be  taken 
not  to  evaluate  the  wrong  criteria.    A  benchmark  test,  for  instance,  tests  the  speed  to 
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perform  a  specific  task,  which  is  really  not  the  most  important  factor  in  a  developing 
scenario.  A  more  appropriate  test  would  be  for  the  correctness  of  the  solution  regarding 
identity  and  recommended  action  and  whether  the  information  presented  without  excessive 
digging  is  the  information  really  needed  by  the  TAO. 

The  critical  issue  here  is  that  a  system  designed  to  assist  the  TAO  be  appropriate  for  his 
environment  and  be  capable  of  providing  him  the  information  he  needs.  The  only  way  to 
accurately  determine  whether  a  design  is  good  enough  is  to  have  fleet  TAOs  try  it  out  in  a 
realistic  environment  and  let  these  end-users  decide  what  it  is  that  they  need. 
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APPENDIX  A 


INTERVIEW  STRUCTURE  FOR  TAO  INTERVIEWS 

DESCRIPTION:  This  appendix  contains  the  questions  and  discussion  points  used  for 
the  structured  interview  technique  to  elicit  the  information  for  the  analysis  and  design  of  the 
Decision  Support  System  (DSS)  for  Tactical  Action  Officers  (TAO).  This  structure  is 
included  to  allow  the  user  to  view  the  actual  questions  and  sequence  of  discussion  areas 
used. 

INTERVIEW  STRUCTURE 

What  ship  types  and  in  what  years  have  you  served  as  a  TAO? 

FF- 1 052  

FFG-7  

Spruance  DD    

Tin  can  

CoontzDDG    

Kidd  DDG       

Amphib  

Auxiliary  

CV/CVN  

CG/CGN         

How  many  years  have  you  been  commissioned? 
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How  many  years  have  you  been  a  Surface  Warfare  Officer? 


What,  if  any,  combat  experience  do  you  have? 


What  is  the  level  of  your  education? 

Bachelors  (Major) Year_ 

Masters  (Major) Year 

Ph.D.  (Major) Year 


How  would  you  characterize  your  background  on  microcomputers  (none, 
minimal,  only  on  a  specific  program,  can  navigate  around  on  them  pretty 
well,  extensive)? 

How  about  other  computers? 

If  a  system  were  available  that  could  help  you  narrow  the  possibilities  about  the 
identities  of  various  contacts  closing  your  ship  (based  on  the  information 
you  or  someone  else  enters  into  it),  would  you  use  it? 


Do  you  think  anyone  else  would? 


How  important  a  help  would  this  be  to  you? 
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Of  this  list  of  information  and  services  the  system  could  provide  to  you,  which 
would  be  the  most  valuable?  The  intent  here  is  to  have  the  subject  rank  the 
items  in  order  of  usefulness  to  him. 

Identity  of  the  Contact 

A  listing  of  the  weapons  each  contact  carries 

A  notification  when  you  are  in  a  particular  contact's  weapons 

release  range 

A  threat  ranking  of  the  various  possible  contacts  in  the  area 

A  memory  aid  to  provide  feedback  on  what  you  and  the 

computer  together  have  determined  a  contact  to  be 

A  reasoning  tool  for  use  in  deciding  on  the  identity  for 

yourself 

An  integrated  package  containing  Rules  of  Engagement 

triggers  and  criteria  for  hostile  intent 

A  weapons  employment  advisor,  to  provide  a  suggestion  about 

which  weapon  and  how  many  of  them  would  have  the  best 
probability  of  taking  out  a  particular  target 

A  simple  EW  advisor  since  this  data  is  acquired  first  and 

farthest  from  the  contact  (this  would  probably  not  provide  a  specific 
identity  except  in  the  case  of  unique  emitters 

Should  the  system  be  modularized  to  allow  viewing  of  only  air  targets,  surface 
targets,  etc.? 
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Or  should  all  warfare  areas  always  be  available? 


Should  the  system  also  have  the  capability  to  permit  the  TAO  to  start  from  an 
educated  guess  (i.e.,  an  intel  report),  then  prompt  the  TAO  for  further  data 
to  confirm  or  deny  the  contact  identification? 


Should  the  search  allow  for  the  pursuit  of  every  possibility  to  prevent  a  false 
identification  or  should  it  permit  the  fastest  possible  conclusion?  Imagine  a 
typical  identification  scenario  for  an  unknown  ship. 


Based  on  the  following  tradeoff  chart,  where  would  you  be  willing  to  have  this 
system  operate  between  certainty  of  the  conclusion  (in  percent)  and  time  to 
figure  (in  minutes)? 
Certainty 
100  % 


50% 


5    8    10 


Time  (MINUTES) 
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Should  the  system  allow  the  used  to  change  already  entered  data  if  it  turns  out  to 
be  wrong  (without  having  to  begin  the  whole  process  all  over  again)  and 
proceed  with  a  new  set  of  rules? 


For  instance,  if  your  EW  information  turned  out  to  be  a  misidentification,  do 
you  want  to  be  able  to  change  it  and  leave  the  rest  of  the  information  the 
same  and  run  the  same  contact? 


Should  the  system  be  platform-specific  to  allow  for  the  various  characteristics  of 
the  different  sensors  (such  as  typical  radar  detection  range)  aboard  the 
various  friendly  ships  which  will  use  such  a  system? 


What  is  the  optimal  number  of  contacts  for  the  system  to  hold  and  keep  track  of 
for  the  TAO  and  computer  to  work  on  together?  Please  keep  in  mind  the 
tactical  situations  in  which  you  found  it  most  difficult  to  keep  track  of  the 
developing  situation. 

5  7  9  11         13         25         50         100 

Why? 

How  would  you  as  a  TAO  find  it  most  beneficial  to  talk  to  your  computer 
assistant? 
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Are  pull-down  menus,  windows,  and  icons  the  answer,  or  will  they  be  as 
intimidating  as  an  MS-DOS-like  text  interface? 


(Here  the  subject  is  shown  the  attached  diagram  of  windows  and  a  brief 

description  of  their  use  is  given  ,  i.e.,  double  click  on  an  icon  to  get  further 
amplifying  data) 


Geographic 
Picture 


1    Potential  Threat 


,,,.,...,,vv,,,,.,,,.,,,,,,,,,,,::,.,,,l,,,., ,,,,,., ,,,,, ,,..,,,,,, 

, 


Current  Status 


3141 

Currently   considering 
Kashin    30% 
Kresta  II      40% 

1147 

Currently    identified 
Skorry     90% 

1137 

Currently   considering 
Moskva  40% 
Kiev    20% 


3141    Weapons  List 

Probable  Activity 

1137    Weapons  List 

Probable  Activity 

1147     Weapons  List 


mmm  wmmmmmMMmMmm& 


How  much  of  a  User's  Manual  would  you  need  to  get  started  with  this 

system  (a  simple  explanation  of  the  menus  and  a  few  examples  of 
scenarios  to  run)? 
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How  detailed  an  instruction  set  should  come  with  the  system?Should  a  real 
live  instructor  come  to  demonstrate  and  advise  for  a  while  with  each 
such  set? 


Should  instruction  for  such  a  system  be  given  at  Department  Head  School? 


How  much?  (none  [get  it  on  the  ship],  brief  introduction,  hands-on  practice) 


If  such  a  system  showed  up  on  your  ship,  would  you  invest  the  time  to  learn  to 
use  it? 


Would  TAOs  become  too  dependent  on  such  a  system  and  be  unable  to  function 
without  it  (in  case  it  went  down)? 


Would  its  use  be  more  similar  to  that  of  NTDS  (if  it's  working  great,  and  if  not, 
operations  go  Mode  IV)? 


Would  this  system  be  a  help  in  replacing  NWP-lookups  for  determining 
which  ships  have  a  certain  radar  combination  or  a  certain  funnel  or  win 
configuration? 
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Should  the  system  require  a  chauffeur  to  free  the  TAO  from  the  interface  problem 
altogether  (other  than  talking  to  the  chauffeur)  and  permit  him  to  focus 
more  thoroughly  on  the  tactical  problem  at  hand? 


Would  the  EWs  be  more  likely  operators  for  this  system  than  the  TAO?  (since 
they  have  the  data  first,  they  could  consult  this  system  and  report  the  results 
to  the  TAO). 


Should  the  information  gleaned  by  the  system  through  processing  the  input  data 
be  automatically  presented  to  the  TAO  every  so  often?        Or  should  the 
information  determined  by  the  system  be  presented  to  the  TAO  only  when 
he  queries  the  system  for  it? 


Should  the  status  display  give  a  measure  of  progress  toward  a  conclusion? 
(Here  the  subject  is  shown  a  series  of  thermometer  sort  of  graphs,  filling 
up  as  the  identity  becomes  more  certain.) 
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What  are  the  critical  ranges  for  such  information  about  certainty? 
0-25  % 
30-40  % 
45-60  % 
65-75  % 
80-90  % 
95  %  and  above 


What  should  be  the  sequence  of  questions  asked  of  the  TAO? 

electronic  warfare 

radar  detection 

visual 

How  often  may  the  TAO  be  queried  for  new  information? 


Should  he  simply  be  able  to  enter  information  in  the  order  he  receives  it  and 
WHEN  he  receives  it? 


Should  a  form  for  each  contact  in  question  be  presented  on  the  screen  (which 
would  look  somewhat  like  a  fill-in-the-blank  form)  into  which  the  TAO 
could  enter  whatever  data  he  has  for  that  contact? 
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Should  this  be  in  response  to  some  action  by  the  TAO  or  chauffeur  (for  instance, 
when  he  gets  new  data),  or  should  it  be  initiated  by  the  system  after  a  certain 
amount  of  time  since  the  generation  of  a  new  contact  or  the  last  update  to 
the  contact? 
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APPENDIX  B 

SUPPLEMENTAL  RESULTS  OF  INTERVIEWS  CONDUCTED 

DESCRIPTION:  This  appendix  contains  the  remainder  of  the  interview  results  not 
discussed  in  the  actual  text  of  this  thesis.  These  data  are  included  to  allow  a  complete 
review  of  the  results  by  the  reader.  This  permits  an  independent  evaluation  of  the  project. 

VALUE  TO  THE  TAO  OF  SERVICES  PROVIDED 

The  data  represented  below  is  the  number  of  incidents  of  a  certain  ranking  for  each  of 
the  services  or  information  the  TAOs  were  asked  to  prioritize  in  value  of  that  particular 
service/information  to  them.  The  numbers  to  the  right  of  the  double  line  indicate  the 
number  of  TAOs  who  assigned  the  service  provided  by  the  system  the  priority  indicated  in 
the  leftmost  column.  For  instance,  four  respondents  indicated  that  the  system's  ability  to 
provide  the  identity  of  the  contact  was  highest  priority  (indicated  by  the  1  in  the  left 
column). 


Priority 
(1-high 

9-low) 

Identity 

of 

Contact 

Weapons 
Listing 

Within 
Weapons 
Release 
Range 

Threat 
Ranking 

Memory 
Aid 

Reasoning 
Tool 

ROE 

Triggers/ 
Hostile 
Intent 

Weapons 
Employ- 
ment 
Advisor 

Simple 

EW 

Advisor 

1 

4 

0 

3 

1 

0 

1 

2 

1 

1 

2 

2 

3 

2 

3 

0 

0 

2 

0 

0 

3 

0 

3 

1 

5 

0 

1 

0 

0 

1 

4 

2 

2 

3 

0 

1 

1 

1 

1 

0 

5 

1 

1 

1 

1 

0 

0 

2 

5 

0 

6 

1 

1 

1 

1 

1 

3 

1 

2 

0 

7 

0 

0 

0 

0 

3 

0 

4 

1 

3 

8 

1 

1 

0 

0 

4 

3 

0 

1 

1 

9 

2 

0 

0 

0 

2 

2 

0 

0 

5 
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APPENDIX  C 


SEVENTEEN  EXPERT  SYSTEMS  SHELLS  INVESTIGATED 
AND  THE  FIVE  FINALISTS 


DESCRIPTION:  The  following  seventeen  names  are  the  expert  systems  which  made 
it  through  the  initial  screening  and  which  were  further  investigated  to  determine  the  five 
finalists,  which  are  listed  below  with  their  corporate  makers  and  the  addresses  of  the 
makers. 

The  seventeen  systems  which  made  it  through  the  initial  screening  and  were  further 
researched  are: 

•  XSYS  by  California  Intelligence 

•  OPS5+  by  Computer  *  Thought  Corporation 

•  ESP  Advisor  by  Expert  Systems  International 

•  Arity  by  Arity  Corporation 

•  Expert  Edge  by  Human  Edge  Software  Co. 

•  Expert-Ease  by  Expert  Systems  Incorporated 

•  EXSYS  by  EXSYS,  Inc. 

•  TTMM  (The  Intelligent  Machine  Model)  by  General  Research 

•  Insight  2+  by  Level  Five  Research,  Inc. 

•  Guru  by  Micro  Data  Base  Systems,  Inc. 
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•  Advisor  by  Ultimate  Media 

•  Insight  by  Jeffrey  Perrone  and  Associates,  Inc. 

•  M.l  by  Teknowledge,  Inc. 

•  SuperExpert  by  Softsync,  Inc. 

•  PC  Scheme  by  Texas  Instruments 

•  Personal  Consultant"1"  by  Texas  Instruments 

•  Goldworks  by  Gold  Hill  Computers,  Inc. 

•  NEXPERT  by  Neuron  Data,  Inc. 

The  five  systems  which  checked  out  for  having  the  ability  to  match  the 
criteria  determined  for  the  design  of  HOTDAM!  are: 


XSYS  by  California  Intellige 
912  Powell  St. 
Suite  8 

San  Francisco,  CA 
94108 

Personal  Consultant"*"  by  Texas  Instruments 
P.  O.  Box  655012,  Mail  Station  57 
Dallas,  TX 
75265 

M.l  by  Teknowledge,  Inc. 
525  University  Ave. 
Palo  Alto,  CA 
94301 

Goldworks  by  Gold  Hill  Computers,  Inc. 
163  Harvard  St. 
Cambridge,  MA 
02139 

NEXPERT  by  Neuron  Data,  Inc. 
444  High  Street 
Palo  Alto,  CA 
94301 
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